klldmofashi commited on
Commit
3915a56
1 Parent(s): 1e573e2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +61 -3
README.md CHANGED
@@ -18,7 +18,7 @@ VILA is a visual language model (VLM) pretrained with interleaved image-text dat
18
  VILA1.5-40b was trained in May 2024.
19
 
20
  **Paper or resources for more information:**
21
- https://github.com/Efficient-Large-Model/VILA
22
 
23
  ```
24
  @misc{lin2023vila,
@@ -40,7 +40,7 @@ https://github.com/Efficient-Large-Model/VILA
40
  - [Dataset Licenses](https://github.com/Efficient-Large-Model/VILA/blob/main/data_prepare/LICENSE) for each one used during training.
41
 
42
  **Where to send questions or comments about the model:**
43
- https://github.com/Efficient-Large-Model/VILA/issues
44
 
45
  ## Intended use
46
  **Primary intended uses:**
@@ -49,8 +49,66 @@ The primary use of VILA is research on large multimodal models and chatbots.
49
  **Primary intended users:**
50
  The primary intended users of the model are researchers and hobbyists in computer vision, natural language processing, machine learning, and artificial intelligence.
51
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
52
  ## Training dataset
53
  See [Dataset Preparation](https://github.com/Efficient-Large-Model/VILA/blob/main/data_prepare/README.md) for more details.
54
 
 
 
 
 
 
 
 
 
 
 
55
  ## Evaluation dataset
56
- A collection of 12 benchmarks, including 5 academic VQA benchmarks and 7 recent benchmarks specifically proposed for instruction-following LMMs.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
18
  VILA1.5-40b was trained in May 2024.
19
 
20
  **Paper or resources for more information:**
21
+ https://github.com/NVLabs/VILA
22
 
23
  ```
24
  @misc{lin2023vila,
 
40
  - [Dataset Licenses](https://github.com/Efficient-Large-Model/VILA/blob/main/data_prepare/LICENSE) for each one used during training.
41
 
42
  **Where to send questions or comments about the model:**
43
+ https://github.com/NVLabs/VILA/issues
44
 
45
  ## Intended use
46
  **Primary intended uses:**
 
49
  **Primary intended users:**
50
  The primary intended users of the model are researchers and hobbyists in computer vision, natural language processing, machine learning, and artificial intelligence.
51
 
52
+ ## Model Architecture:
53
+ **Architecture Type:** Transformer
54
+ **Network Architecture:** InternViT, Yi
55
+
56
+ ## Input:
57
+ **Input Type:** Image, Video, Text
58
+ **Input Format:** Red, Green, Blue; MP4 ;String
59
+ **Input Parameters:** 2D, 3D
60
+
61
+ ## Output:
62
+ **Output Type:** Text
63
+ **Output Format:** String
64
+
65
+ **Supported Hardware Microarchitecture Compatibility:**
66
+ * Ampere
67
+ * Jetson
68
+ * Hopper
69
+ * Lovelace
70
+
71
+ **[Preferred/Supported] Operating System(s):** <br>
72
+ Linux
73
+
74
+ ## Model Version(s):
75
+ * VILA1.5-3B
76
+ * VILA1.5-3B-s2
77
+ * Llama-3-VILA1.5-8B
78
+ * VILA1.5-13B
79
+ * VILA1.5-40B
80
+ * VILA1.5-3B-AWQ
81
+ * VILA1.5-3B-s2-AWQ
82
+ * Llama-3-VILA1.5-8B-AWQ
83
+ * VILA1.5-13B-AWQ
84
+ * VILA1.5-40B-AWQ
85
+
86
  ## Training dataset
87
  See [Dataset Preparation](https://github.com/Efficient-Large-Model/VILA/blob/main/data_prepare/README.md) for more details.
88
 
89
+ ** Data Collection Method by dataset
90
+ * [Hybrid: Automated, Human]
91
+
92
+ ** Labeling Method by dataset
93
+ * [Hybrid: Automated, Human]
94
+
95
+ **Properties (Quantity, Dataset Descriptions, Sensor(s)):**
96
+ 53 million image-text pairs or interleaved image text content.
97
+
98
+
99
  ## Evaluation dataset
100
+ A collection of 12 benchmarks, including 5 academic VQA benchmarks and 7 recent benchmarks specifically proposed for instruction-following LMMs.
101
+
102
+ ## Inference:
103
+ **Engine:** [Tensor(RT), Triton, Or List Other Here]
104
+ * PyTorch
105
+ * TensorRT-LLM
106
+ * TinyChat
107
+
108
+ **Test Hardware:**
109
+ * A100
110
+ * Jetson Orin
111
+ * RTX 4090
112
+
113
+ ## Ethical Considerations
114
+ NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.