[Feature] Blocklist subscriptions plugin with auto scheduled downloads and management UI

[nvandamme]

This PR proposes a new list_subscriptions plugin for OpenSnitch.

The idea is inspired by the existing downloader plugin, but aimed at a more specific use case: managing subscription-based domain lists and integrating them with OpenSnitch rules. The goal is to provide a more complete workflow than a generic periodic downloader, while still fitting into OpenSnitch’s action/plugin model.

Scope

This proposal adds a new plugin with:

• a dedicated configuration UI for subscription management
• support for global defaults and optional per-subscription overrides
• a shared runtime that can be started, stopped, reloaded, and monitored from the UI
• scheduled and manual refresh flows
• asynchronous URL testing and stronger form validation
• atomic locked reads/writes for action files, downloaded files and their derived metadata states
• OpenSnitch rule creation helpers for local-user usage
• rule update signaling when subscribed lists change
• a cleaner storage layout separating source files from rule-facing directories

Warning

This plugin is intended to operate on the local node only.

In short, it is meant to cover the full lifecycle of list subscriptions: configure, download, refresh, expose state, and integrate with rules.

Differences from downloader

The downloader plugin is the starting point of this proposal. It builds on the same general idea while adding a more opinionated workflow around blocklist-style subscriptions:

• named subscriptions instead of a simpler flat downloader config
• inheritance from global defaults
• richer editing and validation
• runtime/UI signaling
• rule-aware file layout
• explicit rule refresh behavior after updates
• local-node-only integration

Current limitations

This proposal intentionally stays conservative in a few areas:

• it only operates on the local node
• rule creation is restricted to the current user
• rule refresh after updates is limited to affected local rules
• startup refresh depends on local node availability and is deferred when needed

These constraints are deliberate for now, to keep the plugin predictable and avoid unintended remote or system-wide side effects.

File storage and linking strategy

The plugin uses a split layout under ~/.config/opensnitch/list_subscriptions:

• real downloaded files are stored under ~/.config/opensnitch/list_subscriptions/sources.list.d/
• rule-facing directories are exposed under ~/.config/opensnitch/list_subscriptions/rules.list.d/

Each subscription downloads its actual list file into:

• ~/.config/opensnitch/list_subscriptions/sources.list.d/<filename>

and stores its metadata file alongside it:

• ~/.config/opensnitch/list_subscriptions/sources.list.d/<filename>.meta.json

These source files are the canonical downloaded artifacts managed by the runtime.

Rules do not consume files directly from sources.list.d. Instead, the plugin builds rule-facing directories in:

• ~/.config/opensnitch/list_subscriptions/rules.list.d/<subscription-dir>
• ~/.config/opensnitch/list_subscriptions/rules.list.d/<group>
• ~/.config/opensnitch/list_subscriptions/rules.list.d/all

Those directories contain symlinks pointing back to the real files in sources.list.d.

This gives the plugin two clearly separated roles:

• sources.list.d is the storage area for downloaded lists and metadata files
• rules.list.d is the clean rule-facing view that OpenSnitch rules can target

UI Design

Main Opensnitch window

The plugin main window is invoked via opensnitch's top action bar (after the firewall icon) or if it failed to inject there it will be available in the application menu.

Plugin main window

Adding/Editing subscriptions

Creating rules from subscriptions

When a rule (either global for all subscribed lists, per user assigned groups, or individually per list if only one is selected) is created (via the available action buttons on the panel), a directory with symlinks to the ordered sources list files is created under ~/.config/opensnitch/rules.list.d/<group/per_list_dir> and directly passed to the new rule dialog.

[gustavo-iniguez-goya]

hi @nvandamme !

This looks impressive!

I think the protobuffer files are not needed, correct me if I'm wrong please. Otherwise, delete them.

And on the other hand, the dialog was refactored not long ago, it's not called stats anymore, but `events:
https://github.com/evilsocket/opensnitch/tree/master/ui/opensnitch/dialogs/events

https://github.com/evilsocket/opensnitch/pull/1561/changes#diff-4a5c5ef370109e533b5d31f49eabca1ac9f4ef969b5a452ed759865da0c378fbR25

it fails with latest master branch.

[nvandamme]

Hi @gustavo-iniguez-goya

Thanks for the quick review.

I've been live testing the plugin against an older branch (opensnitch package from AUR), I'll test against the latest branch for sure!

I'll also remove the proto files : I've just rebuild them to have a pyi annotation file on my side to check types with ruff. I've forgot to delete them in my commits.

Anyway, are the new code paths for events mostly the same as it was for stats? Do you have an example for a plug-in?

[nvandamme]

@gustavo-iniguez-goya seems that event is a drop in replacement of stats, my last commit works against the last opensnitch-git from AUR (direct pull of the last HEAD and build of the Arch package for installing system-wide).

Also regarding protobuf generated files, is it possible to add --pyi_out=../ui/opensnitch/proto/ to generate a python annotation stub that can expose all protobuff classes types and members types for python type checking ?
I've done another PR here : #1564

[gustavo-iniguez-goya]

Perfect @nvandamme ! now it works.

Some notes before merging it:

• When creating a rule, the default duration is Once and it's not enabled, so the rule has no effect. Users will probably expect an enabled permanent rule (Duration: forever).
• When testing a URL from the Subscription editor dialog, if a pop-up is shown (to allow the connection), it's not possible to click in the pop-up dialog.
• Maybe we could hide the nodes combobox for now, to avoid confusions.

Minor things:

• If testing a url fails, the error label expands the Subscription editor dialog:

Future idea:

• Add a list of malware/ad blocklists and let the users subscribe to them, without pasting the URL.

[nvandamme]

@gustavo-iniguez-goya

When creating a rule, the default duration is Once and it's not enabled, so the rule has no effect. Users will probably expect an enabled permanent rule (Duration: forever).

Done, set defaults to forever and enable

When testing a URL from the Subscription editor dialog, if a pop-up is shown (to allow the connection), it's not possible to click in the pop-up dialog.

Done, migrated the url test op to an underlying QThread with a message listener, thus not blocking the main ui thread.

Maybe we could hide the nodes combobox for now, to avoid confusions.

Agreed and done.

If testing a url fails, the error label expands the Subscription editor dialog

Done, keeping short messages in the error status field and stetted up a tooltip that can show the whole error.

[nvandamme]

Add a list of malware/ad blocklists and let the users subscribe to them, without pasting the URL.

It entails to maintain a well curated list of available (and vetted) blocklists, some other projects are already doing that (AdGuard Home, PiHole, Portmaster...).
Shouldn't it be better to maintain a dedicated wiki page and maybe link this page to a permanent issue where willing users can share some sources?
I can then add a direct link to this wiki page inside the plugin.

On thing for future updates that seems interesting, mostly for the capability of handling other lists formats: the Adblock format seems interesting because it can leverage wildcard domain without using regexps.
See the feature request #1563

[nvandamme]

For distant nodes handling and for hardening the plugin runtime (timers schedules, download, lists managing/deployment on distant nodes...), one future update might involve rewriting the runtime inside opensnitch daemon (tough i'm not fluent in go) and mapping the plugin UI with the daemon API/IPC (grpc via new protobuf handlers/defs ?).

[nvandamme]

I've rewritten most of the plugin by adopting a kind of MVC pattern involving underneath QT signals and QT threads to avoid blocking ui/runtime calls.

Some big UI additions:

• Live monitoring of the subscriptions via a new dedicated tab
• An inspect panel to show the actual live state of a selected subscription
• A modal window that list all rules that use a subscription

One big note tough, using directly the underlying opensnitch DB (e.g for rule enumeration or hooking to RulesEditorDialog) is very dangerous because reads are not guarded by any RWLock/mutex and the other core opensnicth calls are fighting to access it (some of my drafts/tests had completely locked the ui and the daemon gRPC side too), hence a careful usage of some differed calls via QtCore.QTimer.singleShot(0,...).

[nvandamme]

@gustavo-iniguez-goya

I've finished hardening the modal that list rule/subscription usage part.

The only good solution i've found to guard against possible race condition accessing the underlying db is to isolate rules enumeration in a separate worker via a three step strategy (isolated each in their own worker thread to avoid ui blocking):

• one worker is query counting existing local node rules
• a second worker is DB querying the rules and their op data
• a third worker is json parsing op fields and matching them in an in memory snapshot cache for cheap afterwards matching between rules and the list subs without doing repeated db queries and the three worker pass.

This cache controller is also connected to opensnitch signals to invalidate any rule entries from add/update/delete ops.

Additionally a simple time-to-process estimate project the whole three worker pass and inform preemptively the user via a modal that this op might take some times (no warning under 2s).

I've also added a live log ui side for fun that is bidirectional (via signals) with the actual logger on the runtime backend.

Some actual screenshots of the last ui: