snzhang commited on
Commit
5208e3d
1 Parent(s): 2388270

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +38 -1
README.md CHANGED
@@ -10,4 +10,41 @@ widget:
10
  - src: >-
11
  https://huggingface.co/snzhang/FileTitle-Beit-GPT2/resolve/main/BorntoFly.jpg
12
  example_title: Born to Fly
13
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
  - src: >-
11
  https://huggingface.co/snzhang/FileTitle-Beit-GPT2/resolve/main/BorntoFly.jpg
12
  example_title: Born to Fly
13
+ ---
14
+
15
+ # Image Caption Model
16
+
17
+ ## Model description
18
+
19
+ The model is used to generate the Chinese title of a random movie post. It is based on the [BEiT](https://huggingface.co/microsoft/beit-base-patch16-224-pt22k-ft22k) and [GPT2](https://huggingface.co/IDEA-CCNL/Wenzhong-GPT2-110M).
20
+
21
+ ## Training Data
22
+
23
+ The training data contains 5043 movie posts and their corresponding Chinese title which are collected by [Movie-Title-Post](https://huggingface.co/datasets/snzhang/Movie-Title-Post)
24
+
25
+ ## How to use
26
+
27
+ ```Python
28
+ from transformers import VisionEncoderDecoderModel, ViTFeatureExtractor, AutoTokenizer
29
+ from PIL import Image
30
+
31
+ pretrained = "snzhang/FileTitle-Beit-GPT2"
32
+ model = VisionEncoderDecoderModel.from_pretrained(pretrained)
33
+ feature_extractor = ViTFeatureExtractor.from_pretrained(pretrained)
34
+ tokenizer = AutoTokenizer.from_pretrained(pretrained)
35
+
36
+ image_path = "your image path"
37
+ image = Image.open(image_path)
38
+ if image.mode != "RGB":
39
+ image = image.convert("RGB")
40
+ pixel_values = feature_extractor(images=image, return_tensors="pt").pixel_values
41
+
42
+ output_ids = model.generate(pixel_values, **gen_kwargs)
43
+ preds = tokenizer.batch_decode(output_ids, skip_special_tokens=True)
44
+ preds = [pred.strip() for pred in preds]
45
+ print(preds)
46
+ ```
47
+
48
+ ## More Details
49
+
50
+ You can get more training details in [FileTitle-Beit-GPT2](https://github.com/h7nian/FileTitle-Beit-GPT2)