| Original | +Translated | +
|---|---|
+
+ 
+
+ + (Source @09ra_19ra) + |
+
+
+ 
+
+ + (Mask) + |
+
+
+ 
+
+ + (Source @VERTIGRIS_ART) + |
+
+
+ 
+
+ + --detector ctd
+ (Mask)
+ |
+
+
+ 
+
+ + (Source @hiduki_yayoi) + |
+
+
+ 
+
+ + --translator none
+ (Mask)
+ |
+
+
+ 
+
+ + (Source @rikak) + |
+
+
+ 
+
+ + (Mask) + |
+
+
+ +```bash +# use `--mode api` to start a web server. +$ python -m manga_translator -v --mode api --use-gpu +# the api will be serving on http://127.0.0.1:5003 +``` + +Api is accepting json(post) and multipart. +
+Api endpoints are `/colorize_translate`, `/inpaint_translate`, `/translate`, `/get_text`. +
+Valid arguments for the api are: + +``` +// These are taken from args.py. For more info see README.md +detector: String +ocr: String +inpainter: String +upscaler: String +translator: String +target_language: String +upscale_ratio: Integer +translator_chain: String +selective_translation: String +attempts: Integer +detection_size: Integer // 1024 => 'S', 1536 => 'M', 2048 => 'L', 2560 => 'X' +text_threshold: Float +box_threshold: Float +unclip_ratio: Float +inpainting_size: Integer +det_rotate: Bool +det_auto_rotate: Bool +det_invert: Bool +det_gamma_correct: Bool +min_text_length: Integer +colorization_size: Integer +denoise_sigma: Integer +mask_dilation_offset: Integer +ignore_bubble: Integer +gpt_config: String +filter_text: String +overlay_type: String + +// These are api specific args +direction: String // {'auto', 'h', 'v'} +base64Images: String //Image in base64 format +image: Multipart // image upload from multipart +url: String // an url string +``` + +
+
+Manual translation replaces machine translation with human translators.
+Basic manual translation demo can be found at API V2
++ +```bash +# use `--mode api` to start a web server. +$ python -m manga_translator -v --mode api --use-gpu +# the api will be serving on http://127.0.0.1:5003 +``` + +Api is accepting json(post) and multipart. +
+Api endpoints are `/colorize_translate`, `/inpaint_translate`, `/translate`, `/get_text`. +
+Valid arguments for the api are: + +``` +// These are taken from args.py. For more info see README.md +detector: String +ocr: String +inpainter: String +upscaler: String +translator: String +target_language: String +upscale_ratio: Integer +translator_chain: String +selective_translation: String +attempts: Integer +detection_size: Integer // 1024 => 'S', 1536 => 'M', 2048 => 'L', 2560 => 'X' +text_threshold: Float +box_threshold: Float +unclip_ratio: Float +inpainting_size: Integer +det_rotate: Bool +det_auto_rotate: Bool +det_invert: Bool +det_gamma_correct: Bool +min_text_length: Integer +colorization_size: Integer +denoise_sigma: Integer +mask_dilation_offset: Integer +ignore_bubble: Integer +gpt_config: String +filter_text: String +overlay_type: String + +// These are api specific args +direction: String // {'auto', 'h', 'v'} +base64Images: String //Image in base64 format +image: Multipart // image upload from multipart +url: String // an url string +``` + +
+
+ +Two modes of translation service are provided by the demo: synchronous mode and asynchronous mode.\ +In synchronous mode your HTTP POST request will finish once the translation task is finished.\ +In asynchronous mode your HTTP POST request will respond with a `task_id` immediately, you can use this `task_id` to +poll for translation task state. + +#### Synchronous mode + +1. POST a form request with form data `file:` to
+2. Wait for response
+3. Use the resultant `task_id` to find translation result in `result/` directory, e.g. using Nginx to expose `result/`
+
+#### Asynchronous mode
+
+1. POST a form request with form data `file:` to
+2. Acquire translation `task_id`
+3. Poll for translation task state by posting JSON `{"taskid": }` to
+4. Translation is finished when the resultant state is either `finished`, `error` or `error-lang`
+5. Find translation result in `result/` directory, e.g. using Nginx to expose `result/`
+
+#### Manual translation
+
+POST a form request with form data `file:` to
+and wait for response.
+
+You will obtain a JSON response like this:
+
+```json
+{
+ "task_id": "12c779c9431f954971cae720eb104499",
+ "status": "pending",
+ "trans_result": [
+ {
+ "s": "☆上司来ちゃった……",
+ "t": ""
+ }
+ ]
+}
+```
+
+Fill in translated texts:
+
+```json
+{
+ "task_id": "12c779c9431f954971cae720eb104499",
+ "status": "pending",
+ "trans_result": [
+ {
+ "s": "☆上司来ちゃった……",
+ "t": "☆Boss is here..."
+ }
+ ]
+}
+```
+
+Post translated JSON to and wait for response.\
+Then you can find the translation result in `result/` directory, e.g. using Nginx to expose `result/`.
+
+
+
+## Next steps
+
+A list of what needs to be done next, you're welcome to contribute.
+
+1. Use diffusion model based inpainting to achieve near perfect result, but this could be much slower.
+2. ~~**IMPORTANT!!!HELP NEEDED!!!** The current text rendering engine is barely usable, we need your help to improve
+ text rendering!~~
+3. Text rendering area is determined by detected text lines, not speech bubbles.\
+ This works for images without speech bubbles, but making it impossible to decide where to put translated English
+ text. I have no idea how to solve this.
+4. [Ryota et al.](https://arxiv.org/abs/2012.14271) proposed using multimodal machine translation, maybe we can add ViT
+ features for building custom NMT models.
+5. Make this project works for video(rewrite code in C++ and use GPU/other hardware NN accelerator).\
+ Used for detecting hard subtitles in videos, generating ass file and remove them completely.
+6. ~~Mask refinement based using non deep learning algorithms, I am currently testing out CRF based algorithm.~~
+7. ~~Angled text region merge is not currently supported~~
+8. Create pip repository
+
+## Support Us
+
+GPU server is not cheap, please consider to donate to us.
+
+- Ko-fi: API
++ +Two modes of translation service are provided by the demo: synchronous mode and asynchronous mode.\ +In synchronous mode your HTTP POST request will finish once the translation task is finished.\ +In asynchronous mode your HTTP POST request will respond with a `task_id` immediately, you can use this `task_id` to +poll for translation task state. + +#### Synchronous mode + +1. POST a form request with form data `file:
| 原始图片 | +翻译后图片 | +
|---|---|
|
+
+
+
+ + (Source @09ra_19ra) + |
+
+
+
+
+ + (Mask) + |
+
|
+
+
+
+ + (Source @VERTIGRIS_ART) + |
+
+
+
+
+ + --detector ctd
+ (Mask)
+ |
+
|
+
+
+
+ + (Source @hiduki_yayoi) + |
+
+
+
+
+ + --translator none
+ (Mask)
+ |
+
|
+
+
+
+ + (Source @rikak) + |
+
+
+
+
+ + (Mask) + |
+