Plugin RFC

[veracioux]

There are some features which could be useful to a lot of people, but we refused to implement because it is out of the scope of the project. Notably, those issues labeled as Plugin. I propose here a plugin system comprising:

• A general C++ API in the form of a library which can be utilized by any language that supports it
• A Python-specific plugin system that would allow plugin creators to leverage the powerful PySide2 bindings (which are officially supported by Qt)

Project structure changes

• The bulk of flameshot should be converted to a shared library called libflameshot, containing basically everything except main.cpp. This requires no major structure changes, just inserting a bunch of FLAMESHOT_EXPORT macros and some tweaks to the build process. The library will reside in this repository.
• The current flameshot executable should essentially just be the current main.cpp linked against libflameshot.
• A libpyflameshot C++ library that will serve as a bridge from C++ to python, defining thin wrapper classes that are adapted to match the python API. This library may even prove as unnecessary as we delve deep into the implementation.
• Add a new repository called pyflameshot that will house the official flameshot python package that is the python API used by plugins.
• In the future we may add a plugins repository that hosts official plugins and/or verified user plugins.

NOTE: There is not much to be said about the C++ API besides just exposing it. I think the python API is a promising especially since we can make use of the officially supported Qt bindings and allow plugin creators to extend flameshot even beyond what the flameshot library API by itself can provision for. I explore the python API in the ensuing sections. Please keep in mind that these ideas are just a brief preliminary taste and an actual implementation will be much more convincing (and I've started working on it).

Loading python plugins

We can divide plugins into two groups:

• system plugins, stored in e.g. /usr/share/flameshot/vendor-plugins/ or the like
• user plugins, stored in $CONFIG_DIRECTORY/plugins/

In each case, a plugin is a python package that resides in one of the aforementioned directories. When flameshot starts, it shall load all plugins from those directories, starting with system plugins first. A plugin is loaded by executing its __init__.py file. The rest is taken care of by the plugin itself.

Plugins can be configured in a "proprietary" way, i.e. each plugin can use its own configuration format and file located where ever the plugin deems fit.

Alternatively, plugins can take their configuration from the usual .ini file, where each plugin gets its config from its own section in the format [plugin.<plugin_name>].

In order to loosen the coupling, improve the landscape for users who version control their configs, and keep things standard, we can decide to have separate files in the $CONFIG_DIRECTORY named as <plugin_name>.ini. If a plugin uses a different config format, a different suffix would be used. (This is my strategy of choice)

Python plugin functionality

High-level commands

All flameshot sub-commands should be available as functions in a pythonic Flameshot class. These functions should have positional args and keyword args (mimicking options). If the flameshot instance is a daemon, the sub-commands should be run as part of the current program. Otherwise, we can make it so that a new flameshot instance is run as a subprocess. One use case is to enable periodic auto-screenshots.

Custom tools

Define a Tool class that plugins can subclass to create their own tools. Tools should have:

• a button icon
• a draw function that maps the underlying pixmap to the screenshot that is obtained by applying the tool over it
• inherit some common attributes/getters/setters like tool size
• each tool should manage its own state as an implementation detail. For example, if the plugin creator implements the straight line tool (ignore the fact that one exists already), they will decide how to store the start and end coordinates.
• other overridable properties like tooltip, shortcut etc which we can decide to add at will, but should keep as generic as possible

Custom final actions

Add an Action class that plugin creators can subclass and define their own behavior. For example, this can be used to define custom upload procedures.

Events

Key events should be available for python plugins to "hook into". We can implement this using sigslots or even as python decorators.

on_start_capture(pixmap: QPixmap, capture_widget: QWidget)

We can discuss the function arguments, which are not that important since they can be queried from the current context.

on_final_action(actions: Actions, pixmap: QPixmap = None)

Called when the capture has finished. Callbacks receive a list of "final actions" that were performed and can selectively cancel some of those actions. They can also modify the pixmap before the actions are executed.

on_capture_finished(success: bool)

Called after the final actions have been performed.

Other events like on_click, etc that may or may not be context specific

Context

The plugins should have a context available which they can use to query the current state of affairs. We can even utilize the recent python ContextVar class that provides a thread-safe context for global variables.

context.py

• flameshot - the current flameshot instance the plugin is loaded into
• capture_context - Mimics the CaptureContext we currently have implemented in C++. Contains information about the original screenshot that is being edited, the active tool, applied tool layers, etc.

Notification API and logger

A thin python wrapper around the notification API and AbstractLogger that we have in C++.

Simple bindings to widgets exposed

The config window, flameshot launcher, and other widgets should be available as QWidgets in the python API so that the plugins can modify them at will. This is made possible by the PySide2 API.

[borgmanJeremy]

Thanks for taking the time to write this. There are somethings in here I really really like which I will highlight below. However, I don't think this is generally the correct direction. It feels like if we were to go that far we might as well do a rewrite in python.

Stated Goals of plugins (IMO)

• Allow parsing of ShareX files so users can leverage ShareX plugin files for custom uploads.
• Support custom editing options so users can write thier own "tools".
• Support custom final actions.
• The API should be a C interface so plugins are language agnostic. As long as the language supports FFI a user can write a plugin in that language.

Concerns

• The scope of this is too large. This makes too many widgets and functions "boundaries" in the API so we will be very restricted in future changes or need to constantly break the API.

• If python is required to run any plugins, we now need to worry about packaging python. @hosiet had some very well written critiques here when I first proposed doing a pure python plugin system. They convinced me that packaging and distributing python cross platform is a huge pain.

• The CLI does not need to be bound to a plugin, it is already scriptable from the command line.

Things I liked

• The event system

  • I had not considered allowing a plugin to essentially register a callback. I really like this idea

• The tool API.

  • I had originally thought the plugin should just return a pixmap and be done, but now that I've seen the way you describe it, your idea is better.
  • With the combination of callbacks this allows us to allow the plugin to handle events like resize, move, undo.

• System and User plugin directions

  • However IMO we should not load all plugins at startup, they should only be loaded when the plugin is called.

Other considerations

• In any plugin system we need a way to handle the plugin crashing. Ideally this would not crash Flameshot proper.

• Where to show tool plugins. Do we allow each tool plugin to have an SVG that is shown in the bar? Or do we just have one "plugin" button that displays a menu and the user chooses a plugin at that time.

[mmahmoudian]

This thread has so far been very enlightening. I learnt few things 🤗 I don't pretend that I've understood everything, but I'm pretty sure I have a solid picture of the proposal and the counter/complementary proposal.

The following are few points that I thought I should share my opinion:

Where to show tool plugins. Do we allow each tool plugin to have an SVG that is shown in the bar? Or do we just have one "plugin" button that displays a menu and the user chooses a plugin at that time

Imho, plugins should be shown like other tools, but with some button texture to show that they are plugins. This would also help new users to know that what they see in screenshots are Flameshot tools and which ones are plugins.

As long as the language supports FFI a user can write a plugin in that language.

I haven't done this before, but based on what I searched, it is fairly easy to do with most languages. That said, I think we should then provide some tangible examples in some popular languages to help people like me get into writing and maintaining plugins.

In general imho the idea is good, and Flameshot can really use it to it's advantage.

[martinszeltins]

If a plugin system gets implemented, I think it should be made very easy for users to add plugins. I think there should be a plugins section where a user can simply enable a plugin instead of having to find plugins on the internet and then downloading them and adding them manually. Perhaps an official plugins repository or something. One-click add plugin.

[mmahmoudian]

I think there should be a plugins section where a user can simply enable a plugin instead of having to find plugins on the internet and then downloading them and adding them manually

If I've understood your comment correctly, I partly disagree and partly agree.

What I agree:
There should be a official plugin repo

What I disagree:
Flameshot should not be responsible to maintaining them (we already have too much on out plate).

Some elaboration on my opinion:

1. It would be nice to have a centralized catalogue so that users can find plugins and potentially install from it.
2. It is a very good idea to have some example functional plugins that Flameshot devs are responsible for maintaining them
3. Flameshot devs should not be maintaining all plugins
4. Flameshot devs should not be held responsible for 3rd-partly plugins
5. Not all the plugins should be shipped with Flameshot

All points above can be achieved easily through a simple repository. There are many repositories with the same exact style such as CRAN (for R), CPAN (for Perl), Melpa (for elisp) and etc. This can be easily achieved by asking plugin devs to submit the tar.gz or zip file of their latest version to our plugin repo. The zip/tar.gz file should have a specific folder structure and should include a file that contains some info including

• plugin name
• version
• author name
• plugin repo url
• commit hash which this version is produced from
• plugin description
• changes in this version

This file can be later parsed by a simple static site generator to populate the repo.

[martinszeltins]

Yes, sorry, I did not mean that all plugins should be oficially maintained by the Flameshot team itself. I was getting at the ease of adding a plugin. The creators of plugins should have a way of submitting their plugins to the official repo. That will make it easy to install them for users.

Plugins should be maintained by those who create them. They should be responsible for keeping them up to date as well.

[jack9603301]

The core of a plugin is the plugin manager, whose design determines how a plugin is implemented and the specifications that the plugin should follow!

[jack9603301]

@veracioux The first plugin RFC initial implementation will only support the C++ Qt plugin framework, but will leave room for full extensibility.

[jack9603301]

PR #3661

[gander]

Bump