hirol commited on
Commit
fe85af9
•
1 Parent(s): e0787f7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +34 -0
README.md CHANGED
@@ -1,3 +1,37 @@
1
  ---
 
 
2
  license: mit
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: ControlnetWithBackground
3
+ emoji: 🌖
4
  license: mit
5
  ---
6
+ Here is model, add character skeletons to a forest scene generated by SD, and keep the background unchanged to generate controllable characters
7
+
8
+ https://huggingface.co/spaces/hirol/controlnetOverMask
9
+
10
+ # ControlnetWithBackground
11
+ ### Controlnet
12
+ ### Inpainting
13
+ ### Stable diffusion
14
+
15
+ Use controlnet to generate sketches and characters, and keep the background unchanged.
16
+
17
+ Usually when we generate a very perfect background image, we want to add image elements, but using controlnet directly will affect the original background. This project aims to add elements to the page while keeping the background unchanged, and can directly operate on the original background.
18
+
19
+ Support skeletal character generation and sketch generation.
20
+
21
+ Optimize an inpaint model for the general domain against the stablediffusion-inpaint model.
22
+
23
+
24
+ ``` python
25
+ from diffusers import StableDiffusionControlNetPipeline, ControlNetModel, UniPCMultistepScheduler
26
+
27
+ controlnet = ControlNetModel.from_pretrained("./models/control_any5_openpose", torch_dtype=torch.float16)
28
+ ```
29
+
30
+
31
+ ### Two modes, openpose control and manuscript control
32
+ #### openpose control
33
+ Add character skeletons to a forest scene generated by SD, and keep the background unchanged to generate controllable characters
34
+ <img src="https://huggingface.co/spaces/hirol/ControlnetWithBackground/resolve/main/person.png" width="400" height="300">
35
+ <video width="320" height="240" controls>
36
+ <source src="https://huggingface.co/spaces/hirol/ControlnetWithBackground/resolve/main/person_control.mp4" type="video/mp4">
37
+ </video>