Obsidian-Excalidraw 1.8.0 - OCR, Optical Character Recognition of handwritten text and images
Based on Zsolt's Visual Personal Knowledge Management's video on YouTube. If you like this content, support the original creators by watching, liking and subscribing to their content.
Enable OCR in Excalidraw plugin settings under Experimental Features to access Taskbone-powered recognition.
Briefing
A new Obsidian plugin adds experimental OCR for handwritten notes and images, letting users search their scanned Free Draw content inside Obsidian. The key shift is that text written with Excalidraw’s Free Draw tool—or captured from images such as a photo of a blackboard—can be recognized by a cloud service and then inserted into the note so it becomes searchable.
The OCR capability is powered by Taskbone, a third-party cloud provider. When a user triggers OCR, the plugin sends the selected image content to Taskbone’s servers, where Google-based infrastructure performs the recognition. The resulting text is returned to Obsidian and placed both on the clipboard (for pasting into a text element) and into the drawing’s front matter (so it can be indexed and found through Obsidian search). Because processing happens in the cloud, users who consider data transfer a deal-breaker are advised not to enable the plugin. Enabling the feature also requires accepting Taskbone’s terms and privacy policy, with more details available at taskbone.com.
The feature is labeled experimental and released in a “minimum viable product” state, meaning it works but lacks automation. Users must manually press a scan button to initiate OCR; there’s no automatic scanning of documents. To use it, the plugin must be enabled in Obsidian settings under the Excalidraw plugin’s experimental features, where the Taskbone OCR option appears.
OCR scope is intentionally narrow. The plugin captures only Free Draw elements and pictures; it does not scan other drawing elements. For example, shapes like rectangles—even if they cover parts of handwritten text—or text elements already present in the markdown are ignored. The rationale is practical: text elements are already stored as markdown, so OCR would add no value.
Once an image is processed, the drawing shows that it has already been handled. If users need to re-run recognition, they can rescan by holding the Control key (Command on macOS) while pressing the scan button. The scan typically takes a few seconds, after which a message confirms that recognized text was placed on the clipboard and in the note’s front matter.
The payoff is searchability. Recognized handwriting from a blackboard/whiteboard image becomes searchable via Obsidian’s search function, returning the drawing note when matching terms appear in the OCR output. The workflow effectively closes a gap highlighted in PC Magazine’s review of Obsidian: the lack of built-in handwriting recognition and the ability to make scanned content searchable within the Vault.
Pricing is not yet established. The service currently has no price, and it’s expected to remain free for a reasonable number of scans, but frequent use may require a paid version with a license key and API key. Users can also report issues or ideas via GitHub, including links to Taskbone’s GitHub for direct feedback to the OCR service developer.
Cornell Notes
Obsidian’s Excalidraw integration now includes experimental OCR that turns handwritten Free Draw strokes and images (like photos of a blackboard) into searchable text. Recognition is performed by Taskbone, a cloud service, and the resulting text is written to both the clipboard and the drawing’s front matter so Obsidian search can index it. The feature is manual (no auto-scan) and only processes pictures plus Free Draw elements—rectangles and existing text elements are ignored. Because images are sent to the cloud, enabling the plugin requires accepting Taskbone’s terms and privacy policy. The service is currently free, but a paid tier may arrive for heavy usage.
What exactly gets OCR’d, and what gets ignored in Excalidraw drawings?
Where does the recognized text go after scanning?
How does the OCR process work behind the scenes, and what are the privacy implications?
How does a user trigger OCR, and how can they rescan content?
How does OCR improve day-to-day use in Obsidian?
What does “experimental” mean for this OCR feature?
Review Questions
- Which Excalidraw elements are eligible for OCR, and why are some elements excluded?
- How does OCR output get indexed so that Obsidian search can find handwritten text later?
- What user action is required to start OCR, and what modifier key enables rescanning?
Key Points
- 1
Enable OCR in Excalidraw plugin settings under Experimental Features to access Taskbone-powered recognition.
- 2
OCR processes only Free Draw strokes and pictures; shapes like rectangles and existing text elements are ignored.
- 3
Trigger recognition manually via the OCR scan button; there is no automatic scanning.
- 4
Recognized text is written to both the clipboard and the drawing’s front matter, enabling Obsidian search.
- 5
OCR runs in the cloud via Taskbone, so enabling the plugin requires accepting Taskbone’s terms and privacy policy.
- 6
Rescan an already-processed drawing by holding Control (Command on macOS) while pressing the scan button.
- 7
Pricing is currently free, but a paid tier may be introduced for frequent use with a license/API key.