Plugins

Built-in plugins

This is a list of built-in plugins that are considered stable.

See the Plugins section of the user guide for details on how built-in plugins are loaded.

reader.enclosure_dedupe

Deduplicate the enclosures of an entry by enclosure URL.

reader.entry_dedupe

Deduplicate the entries of a feed.

Sometimes, the format of the entry id changes for all the entries in a feed, for example from example.com/123 to example.com/entry. Because the entry id is used to uniquely identify entries, normally this results in the entry being added again with the new id.

This plugin addresses this by copying entry user attributes like read or important from the old entry to the new one.

Note

There are plans to delete the old entry after copying user attributes; please +1 / comment in #140 if you need this.

Duplicates are entries with the same title and the same summary/content.

By default, this plugin runs only for newly-added entries. To run it for the existing entries of a feed, add the .reader.dedupe.once tag to the feed; the plugin will run on the next feed update, and remove the tag afterwards. To run it for the existing entries in a feed, and only use the title for comparisons (ignoring the content), use .reader.dedupe.once.title instead.

Entry user attributes are set as follows:

read

If the old entry is read, the new one will be too. If the old entry is unread, it will be marked as read in favor of the new one.

before

after

old.read

old.read

new.read

True

True

True

False

True

False

important

If the old entry is important, it will be marked as unimporant, and the new one will be marked as important.

before

after

old.important

old.important

new.important

True

False

True

False

False

False

To reduce false negatives when detecting duplicates:

  • All comparisons are case-insensitive, with HTML tags, HTML entities, punctuation, and whitespace removed.

  • For entries with content of different lengths, only a prefix of common (smaller) length is used in comparison. (This is useful when one version of an entry has only the first paragraph of the article, but the other has the whole article.)

  • For entries with longer content (over ~48 words), approximate matching is used instead of an exact match (currently, Jaccard similarity of 4-grams).

To reduce false positives when detecting duplicates:

  • Titles must match exactly (after clean-up).

  • Both entries must have title and content.

  • Similarity thresholds are set relatively high, and higher for shorter content.

reader.mark_as_read

Mark added entries of specific feeds as read if their title matches a regex.

To configure, set the make_reader_reserved_name('mark_as_read') (by default, .reader.mark_as_read) feed metadata to something like:

{
    "title": ["first-regex", "second-regex"]
}

reader.ua_fallback

Retry feed requests that get 403 Forbidden with a different user agent.

Sometimes, servers blocks requests coming from reader based on the user agent. This plugin retries the request with feedparser’s user agent, which seems to be more widely accepted.

Servers/CDNs known to not accept the reader UA: Cloudflare, WP Engine.

Loading plugins from the CLI and the web application

There is experimental support of plugins in the CLI and the web application.

Warning

The plugin system/hooks are not stable yet and may change without any notice.

To load plugins, set the READER_PLUGIN environment variable to the plugin entry point (e.g. package.module:entry_point); multiple entry points should be separated by one space:

READER_PLUGIN='first.plugin:entry_point second_plugin:main' \
python -m reader some-command

To load web application plugins, set the READER_APP_PLUGIN environment variable in a similar way.

For built-in plugins, it is enough to use the plugin name (reader.XYZ).

Note

make_reader() ignores the plugin environment variables.

Experimental plugins

reader also ships with a number of experimental plugins.

For these, the full entry point must be specified.

To use them from within Python code, use the entry point as a custom plugin:

>>> from reader._plugins import sqlite_releases
>>> reader = make_reader("db.sqlite", plugins=[sqlite_releases.init])

tumblr_gdpr

Accept Tumblr GDPR stuff.

Since May 2018, Tumblr redirects all new sessions to an “accept the terms of service” page, including RSS feeds (supposed to be machine-readable), breaking them.

This plugin “accepts the terms of service” on your behalf.

To load:

READER_PLUGIN='reader._plugins.tumblr_gdpr:tumblr_gdpr' \
python -m reader update -v

Implemented for https://github.com/lemon24/reader/issues/67.

Note

This plugin does not seem to be needed anymore as of August 2020.

enclosure_tags

Fix tags for MP3 enclosures (e.g. podcasts).

Adds a “with tags” link to a version of the file with tags set as follows:

  • the entry title as title

  • the feed title as album

  • the entry/feed author as author

This plugin needs additional dependencies, use the unstable-plugins extra to install them:

pip install reader[unstable-plugins]

To load:

READER_APP_PLUGIN='reader._plugins.enclosure_tags:init' \
python -m reader serve

Implemented for https://github.com/lemon24/reader/issues/50. Became a plugin in https://github.com/lemon24/reader/issues/52.

preview_feed_list

If the feed to be previewed is not actually a feed, show a list of feeds linked from that URL (if any).

This plugin needs additional dependencies, use the unstable-plugins extra to install them:

pip install reader[unstable-plugins]

To load:

READER_APP_PLUGIN='reader._plugins.preview_feed_list:init' \
python -m reader serve

Implemented for https://github.com/lemon24/reader/issues/150.

sqlite_releases

Create a feed out of the SQLite release history pages at:

Also serves as an example of how to write custom parsers.

This plugin needs additional dependencies, use the unstable-plugins extra to install them:

pip install reader[unstable-plugins]

To load:

READER_PLUGIN='reader._plugins.sqlite_releases:init' \
python -m reader serve