Here's how we made a no-fuss RSS vulture app using trendy Electron

Everyone's using this framework – now you can, too. Source is open

Code corner Keeping up with the endless torrent of stuff happening online is a losing battle. In the absence of omniscience, there's just no way to catch every bug, cockup, blowup, and scandal as it breaks.

The next best thing is RSS, the aging syndication protocol that provides a convenient mechanism for aggregating online content.

In the course of my work for The Register, as well as meeting and interviewing folks in the tech industry, and poring over documentation and lawsuits, I'm constantly scouring online sources to keep on top of chatter in the field, and daily, if not hourly, developments.

RSS feeds, available from many websites, forums, blogs, government agencies, and aggregators, can be collected in an RSS-reading app to create a sort of personalized homepage of the internet, but none of the RSS apps I'm aware of provide the experience I wanted.

First, the app has to be oriented toward the efficient display of text, to maximize the number of headlines visible. It should be a headline scanning app, not a news reading app. It should not display images at all, because they take up screen space and slow load times.

Second, the app should sort stories chronologically. That's something more RSS apps should do.

Third, the application should filter by time. It should contain links to new pages and not old tired stuff. That means it shouldn't retain material, which most RSS readers seem to do. There should be no concept of "read" or "unread" articles published within a specified time window. I don't want to open it up after a week and find thousands of unread pages.

Fourth, it should parse news aggregators for source links, taking you straight to the original reporting.

Fifth, it should be free, open source, and have no ads.

Unaware of anything that met all my requirements, I created a custom RSS headline scanning app for journalists and news junkies using the Electron framework. It's a desktop app called vulture-feeds and it's now available via GitHub under the open-source MIT license to anyone who may be interested.

Electron

Electron just reached version 2.0 as a beta release. It's a tool for building cross-platform (macOS, Linux, Windows) desktop apps using web technologies – JavaScript, HTML, and CSS, or maybe TypeScript if you're sold on that sort of thing. You should be able to design and write your application once, and it works across all three operating systems.

Vultures' nest ... Screenshot of Tom's custom RSS reader

Professional developers tend to scoff at Electron apps, and rightly so. Electron apps simply aren't as fast or capable as native platform code. They tend to be bloated and to consume quite a bit of memory. Last time I checked, vulture-feeds weighed in at 368.9 MB. Well-written native macOS apps like Sublime Text have more functionality and require a tenth of the space.

Predictably, a recent discussion of the release on Hacker News highlighted the known downsides of Electron and cross-platform apps in general.

Former Apple CEO and co-founder Steve Jobs summed up the problems in his 2010 dismissal of Adobe Flash on iOS: "[D]evelopers only have access to the lowest common denominator set of features."

Cross-platform apps have got a bit better since then – Microsoft's Xamarin has a lot going for it, and Flutter looks promising – but the basic trade-offs remain: native apps can be faster, lighter, and more conformant to platform paradigms; cross-platform apps offer a shortcut and are easier to maintain than several native codebases.

If I were a full-time professional developer, I might choose something other than Electron. But as an occasional coder, I'm grateful there's a way to turn my modest knowledge of JavaScript, Node.js and the like into a functional desktop application.

For what it's worth, organizations and projects from Microsoft, with its Visual Studio Code, to Slack, WhatsApp, and Signal, have made a similar decision to develop desktop apps using Electron.

vulture-feeds

I use vulture-feeds on macOS but Electron allows apps to be built for multiple platforms, so it should also work on Linux and Windows. I haven't tested it on either of these, so your mileage may vary.

I probably could have written the app in Node.js, and hosted it on a website sitting on a server, like a Digital Ocean droplet, somewhere, but then I'd be paying fees unnecessarily. I don't see any reason to put the app into the cloud as it only fetches data on demand. That's something a local machine can handle.

The code base won't win any awards. I haven't written any significant web apps before, so I expect vulture-feeds almost certainly contains suboptimal design and implementation decisions. Feel free to submit pull requests to improve it.

Despite its flaws, it works for me. It makes my job as a reporter a bit easier. And when it doesn't, I can always improve it.

How it works

The app relies on the NPM module feedparser, among others, to fetch RSS/Atom feeds chosen by the user. Clicking on the listed links will open the article in your browser of choice.

Upon startup, the main window should be blank. Add a new RSS/ATOM feed via File -> Add Feed or the handful of feeds in the default set using File->Load Defaults. The Reload command updates everything.

Fetched feeds are filtered and put into an HTML table, after which they can be sorted by publication time. The time window for stories can also be adjusted from the default of 24 hours to has high as 72 hours or as low as 1 hour. Stories published outside the time window won't be displayed.

Because RSS and ATOM feeds can be quirky, I had to hard-code some feed-specific handlers to normalize the articles and their links, for things like the ArXiv computer science feed. Users of the app may want to make similar accommodations in the code if they find a newly added feed misbehaves.

The app can also be used to check if a specific web page has changed, a feature I use for monitoring websites that don't have an RSS feed. It supports two methods of change detection: diffing page checksums and diffing link checksums. Neither is completely reliable because reliable webpage change detection is difficult.

Watched pages of this sort appear in the article list with article links, but start with a ">" character to differentiate them. Watched pages should appear at the top of the article list after sorting.

Using the menu commands Show Feeds and Show Watched Pages, feeds and watched pages can be deleted and their appearance on the main page can be altered.

The app will export the IndexDB/Dexie database of feeds as a json file named "vfdb.json," a feature I added so I can transfer feed lists between home and work machines. I hard-coded that name rather than deal with the various sanitation issues that arise when you let users choose filenames. Maybe I'll fix that at a later date.

A better way to do this would be to transfer the data through an online account, but I'm not presently planning to develop a backend.

I've left the Developer Tools menu active because it's often useful to see what's going on behind the scenes.

So that's it. I hope someone else finds vulture-feeds useful – and, cough, don't forget to point it at El Reg's feeds. ®