Obsidian launched safety scorecards for every plugin, and checking mine changed what I keep installed

If you're a long-time Obsidian user, I probably don't have to tell you that plugins have a habit of piling up. You see one recommendation in a Reddit thread here, another in a YouTube tutorial there, and before you know it you have eight community plugins that you haven't touched in months, and the question of who's maintaining it or what it can actually reach in your vault doesn't come up again.

Most of the time, it's probably fine, but plugins are still third-party code running inside an app we trust with our personal notes, which can be of concern especially since many of them are now vibe-coded. To users' benefit, Obsidian has been running automated checks on every one of them since May and posting the results online. This doesn't show up in the app and instead you have to look for it in a browser.

What an Obsidian plugin scorecard actually checks

It's the same catalog as the app

Obsidian Community is a web directory that went live in May, and it's the same catalog you can scroll in the app's plugin browser, just with a lot more detail about every plugin. Before it launched, Obsidian's small team reviewed a plugin by hand when it was first submitted and didn't review later versions at all, which means that something that got approved years ago could have changed and no one checked. Now, an automated system scans every new version, which would ideally have been the case from the start, though I get that a smaller team couldn't keep up with everything.

With this new system, every plugin has a Scorecard tab and a sidebar that sums it up with two bars. The first one is Health, and it's about whether anyone's still looking after the plugin, and it also pulls in recent commits and install numbers. My Excalidraw plugin, for example, shows a commit five days ago and 97.5K installs. Below that is the Review bar, which scans the latest release and checks for vulnerable dependencies and whether the published build can be reproduced byte-for-byte from the source.

Then further down is Disclosures and Other, where the scorecard spells out what a plugin can reach. Excalidraw's says it reads the system clipboard and can see every file path in the vault, for example, and this isn't something I considered about a drawing plugin. Every scorecard also carries a work-in-progress banner saying it might contain errors.

Reading my own scorecards without panicking

The stuff that seems alarming is not that alarming

I've cycled through dozens of Obsidian plugins across my multiple vaults, but always end up coming back to the basics. One of my vaults currently only has four community plugins, which is a pretty short list and felt like it would be a quick job, until one of the scorecards threw the word "malware" at me. However, all four of mine say Malware scan not available, and the same line turns up on big plugins like Dataview and Templater too. As far as I can tell, it just means there’s no malware result to show for anyone yet, so it isn’t a mark against my plugins, though it also means none of these scorecards can tell me a plugin is malware-free.

I was a little nervous about Excalidraw because it uses base64 encoding, which the scorecard says is sometimes used to hide things like web addresses or API keys in a plugin’s code. But further down you could see that Obsidian rebuilt the plugin from its public source code and got a file identical to the one I’d installed, so the one in my vault matches code that anyone can go and read. The fact that it has 8.5 million downloads also helps, since I presume that many people would have spotted anything dodgy by now.

Another thing I recommend to look for is when Health and Review disagree. The Text Generator plugin had excellent Health but got "caution" for Review, and its listing still says it uses GPT-3. I've had it disabled for ages now, so it was actually a pretty easy uninstall. Screen Pet Companion got a satisfactory review, but with one release and around 450 installs it kind of comes across as a side project - I kept it anyway because I just think it's cute. The JokerType plugin passed everything with flying colors, but I ended up getting rid of it anyway because I haven't used it much.

The cutoff that the app doesn't mention

Failing plugins are about to disappear from the in-app browser

Separately from the scorecards, Obsidian told developers that scans on existing plugins are only informational until October 30. After that, a plugin whose latest release still fails review, which means actual errors rather than a Caution rating, won’t be listed in the directory or installable in the app. This information is not in the Obsidian app, and the installed plugins still just have their names and descriptions.

The score also only covers the code, so they're not going to be able to tell you if a plugin makes sense for your vault. This is why I ended up with a boring rule where I remove a plugin if it's been disabled for a few months.

My vault had more leftovers than I realized

My plugins didn't turn out to be doing anything sketchy, which is obviously a good thing, but it also would have been a good experience to come across something that's sketchy, just to see how the scorecard tool flags it. But I still walked away with a cleaner setup since this check forced me to reevaluate my plugin list and get rid of some of them. The October 30 date is close, too, for anyone who's concerned about their own plugins.