Get AI summaries of any video or article — Sign up free
Obsidian-Excalidraw 1.8.0 - OCR, Optical Character Recognition of handwritten text and images thumbnail

Obsidian-Excalidraw 1.8.0 - OCR, Optical Character Recognition of handwritten text and images

4 min read

Based on Zsolt's Visual Personal Knowledge Management's video on YouTube. If you like this content, support the original creators by watching, liking and subscribing to their content.

TL;DR

Enable OCR in Excalidraw plugin settings under Experimental Features to access Taskbone-powered recognition.

Briefing

A new Obsidian plugin adds experimental OCR for handwritten notes and images, letting users search their scanned Free Draw content inside Obsidian. The key shift is that text written with Excalidraw’s Free Draw tool—or captured from images such as a photo of a blackboard—can be recognized by a cloud service and then inserted into the note so it becomes searchable.

The OCR capability is powered by Taskbone, a third-party cloud provider. When a user triggers OCR, the plugin sends the selected image content to Taskbone’s servers, where Google-based infrastructure performs the recognition. The resulting text is returned to Obsidian and placed both on the clipboard (for pasting into a text element) and into the drawing’s front matter (so it can be indexed and found through Obsidian search). Because processing happens in the cloud, users who consider data transfer a deal-breaker are advised not to enable the plugin. Enabling the feature also requires accepting Taskbone’s terms and privacy policy, with more details available at taskbone.com.

The feature is labeled experimental and released in a “minimum viable product” state, meaning it works but lacks automation. Users must manually press a scan button to initiate OCR; there’s no automatic scanning of documents. To use it, the plugin must be enabled in Obsidian settings under the Excalidraw plugin’s experimental features, where the Taskbone OCR option appears.

OCR scope is intentionally narrow. The plugin captures only Free Draw elements and pictures; it does not scan other drawing elements. For example, shapes like rectangles—even if they cover parts of handwritten text—or text elements already present in the markdown are ignored. The rationale is practical: text elements are already stored as markdown, so OCR would add no value.

Once an image is processed, the drawing shows that it has already been handled. If users need to re-run recognition, they can rescan by holding the Control key (Command on macOS) while pressing the scan button. The scan typically takes a few seconds, after which a message confirms that recognized text was placed on the clipboard and in the note’s front matter.

The payoff is searchability. Recognized handwriting from a blackboard/whiteboard image becomes searchable via Obsidian’s search function, returning the drawing note when matching terms appear in the OCR output. The workflow effectively closes a gap highlighted in PC Magazine’s review of Obsidian: the lack of built-in handwriting recognition and the ability to make scanned content searchable within the Vault.

Pricing is not yet established. The service currently has no price, and it’s expected to remain free for a reasonable number of scans, but frequent use may require a paid version with a license key and API key. Users can also report issues or ideas via GitHub, including links to Taskbone’s GitHub for direct feedback to the OCR service developer.

Cornell Notes

Obsidian’s Excalidraw integration now includes experimental OCR that turns handwritten Free Draw strokes and images (like photos of a blackboard) into searchable text. Recognition is performed by Taskbone, a cloud service, and the resulting text is written to both the clipboard and the drawing’s front matter so Obsidian search can index it. The feature is manual (no auto-scan) and only processes pictures plus Free Draw elements—rectangles and existing text elements are ignored. Because images are sent to the cloud, enabling the plugin requires accepting Taskbone’s terms and privacy policy. The service is currently free, but a paid tier may arrive for heavy usage.

What exactly gets OCR’d, and what gets ignored in Excalidraw drawings?

OCR targets only Excalidraw Free Draw elements and pictures. Other drawing elements—like shapes such as rectangles that may cover parts of handwriting—are disregarded. Text elements already present in the note (markdown text) are also not scanned because they’re already machine-readable.

Where does the recognized text go after scanning?

After OCR runs, the recognized text is placed on the clipboard so it can be pasted into a text element. It’s also inserted into the drawing’s front matter (as a Taskbone OCR-related tag/field), which allows Obsidian’s search to find the content later.

How does the OCR process work behind the scenes, and what are the privacy implications?

The plugin sends the image content to Taskbone’s cloud servers for recognition, with the processing happening on Google servers. Since data is transmitted to a third party, users who don’t want cloud processing should avoid enabling the plugin. Enabling also requires agreeing to Taskbone’s terms and privacy policy.

How does a user trigger OCR, and how can they rescan content?

OCR is initiated manually by pressing the scan button in the Obsidian panel (the OCR grab icon). If the drawing has already been processed, users can rescan by holding Control (Command on macOS) while pressing the scan button. The scan takes a few seconds and then confirms completion.

How does OCR improve day-to-day use in Obsidian?

It makes handwritten content searchable. After OCR inserts text into front matter, Obsidian search can match terms found in the recognized handwriting. Searching for a phrase like “Optical recognition” or “strengthening tools” can return the drawing note containing the OCR output.

What does “experimental” mean for this OCR feature?

It’s a minimum viable product: it works but isn’t fully polished. The workflow requires manual scanning (no automatic scanning), and the feature is enabled through plugin settings under experimental features. Users should expect rough edges while it matures.

Review Questions

  1. Which Excalidraw elements are eligible for OCR, and why are some elements excluded?
  2. How does OCR output get indexed so that Obsidian search can find handwritten text later?
  3. What user action is required to start OCR, and what modifier key enables rescanning?

Key Points

  1. 1

    Enable OCR in Excalidraw plugin settings under Experimental Features to access Taskbone-powered recognition.

  2. 2

    OCR processes only Free Draw strokes and pictures; shapes like rectangles and existing text elements are ignored.

  3. 3

    Trigger recognition manually via the OCR scan button; there is no automatic scanning.

  4. 4

    Recognized text is written to both the clipboard and the drawing’s front matter, enabling Obsidian search.

  5. 5

    OCR runs in the cloud via Taskbone, so enabling the plugin requires accepting Taskbone’s terms and privacy policy.

  6. 6

    Rescan an already-processed drawing by holding Control (Command on macOS) while pressing the scan button.

  7. 7

    Pricing is currently free, but a paid tier may be introduced for frequent use with a license/API key.

Highlights

Handwritten Free Draw and image content can be converted into text that Obsidian can search.
OCR results land in two places: the clipboard for copy/paste and the drawing’s front matter for indexing.
Cloud processing is central: images are sent to Taskbone servers for recognition, so privacy considerations matter.
The feature is intentionally limited—rectangles and existing text elements won’t be OCR’d.
Manual scanning is required, reflecting an experimental, minimum-viable-product stage.

Topics

  • Obsidian OCR
  • Excalidraw Free Draw
  • Taskbone Cloud Recognition
  • Searchable Handwriting
  • Experimental Plugin Setup

Mentioned

  • OCR