The page is in Japanese or English. The app finds the speech bubbles, reads the text and draws the translation on top of them. The model runs on your own machine, so the page never leaves for anyone's server.
Speech bubbles are located with computer vision. A long webtoon strip does not fit the detector in one frame, so it is cut into overlapping slices and the boxes are offset back into place. A bubble split by a seam is merged again.
Each bubble is read on its own with the right engine: manga-ocr on the GPU for Japanese, EasyOCR for English. Reading order is either top down (webtoon) or band by band right to left (manga).
Every line on the page goes to the model in a single request, so context carries from one bubble to the next. A description of the series written into the settings (names, genders, speech styles) rides along and keeps the translation consistent.
The translation is drawn over the bubbles as a click-through layer, so the page still scrolls normally. Read the same image twice and the result comes from cache. The original file is never touched.
┌─ desktop app screen capture, PyQt
pipeline ─────────┼─ local server ← Chrome extension
detect → read └─ cloud worker ← control plane
→ translateThe translation chain is written once and all three entry points call the same function. The desktop app captures a selected region of the screen, the extension hands over the page's own image at full resolution, and the cloud worker takes jobs off a queue.
The control plane (identity, quota, queue) holds no model at all, so it runs on cheap CPU. The heavy work happens on a GPU, and because the worker pulls jobs itself, no port or tunnel has to be opened into the home network.
The extension does not take a screenshot. It processes the page's own <img> at full resolution. The reader is clearly better on it, screen density and scrolling stop mattering, and the image address is a natural cache key.
Site permissions are not requested at install time but only when the user wants to run the extension on a given page. The extension therefore never sees pages where it was not asked for.
The translation model was picked by measurement, not by impression: 30 real pages, 576 lines, 40 of them scored blind. The winner was also the faster one and the one whose license actually allows the use, which the previous default did not. The same method settled the bubble detector too.
Next to the translation there is a correction button. What the user fixes is written to a file, and that file is the dataset a model of my own is later trained on.
malli / model pisteet / score aika / time
gemma3:4b 3.42 103 s
aya-expanse:8b 3.23 130 s
qwen3:8b 3.12 2243 s
qwen2.5:7b 2.62 117 s
mistral:7b 1.65 145 s
30 sivua · 576 repliikkiä · 40 sokkopisteytystä
30 pages · 576 lines · 40 scored blindScoring was done with the model names hidden, and the decision rule was written down before the run.
For a reader following a series that has not been translated into their language. The target language is chosen in the settings: Turkish, Finnish or English.
The same engine also translates live Twitch chat: you frame the chat column on screen and the messages come back translated, with no account and no connection to the service. A single message can also be tapped for translation.
Image and text stay on the machine, and the original work is neither modified nor passed on: the translation is a layer on the reader's own screen.
Text out of an image, a translation out of the text, the translation back on top of the image, all on your own hardware without the material passing through an outside server. If you need the same chain for some other content, tell me what you are building.