Divyasreepat commited on
Commit
b09a656
1 Parent(s): 2c68b0f

Update README.md with new model card content

Browse files
Files changed (1) hide show
  1. README.md +140 -14
README.md CHANGED
@@ -1,17 +1,143 @@
1
  ---
2
  library_name: keras-hub
3
  ---
4
- This is a [`Bert` model](https://keras.io/api/keras_hub/models/bert) uploaded using the KerasHub library and can be used with JAX, TensorFlow, and PyTorch backends.
5
- Model config:
6
- * **name:** bert_backbone
7
- * **trainable:** True
8
- * **vocabulary_size:** 30522
9
- * **num_layers:** 24
10
- * **num_heads:** 16
11
- * **hidden_dim:** 1024
12
- * **intermediate_dim:** 4096
13
- * **dropout:** 0.1
14
- * **max_sequence_length:** 512
15
- * **num_segments:** 2
16
-
17
- This model card has been generated automatically and should be completed by the model author. See [Model Cards documentation](https://huggingface.co/docs/hub/model-cards) for more information.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  library_name: keras-hub
3
  ---
4
+ ### Model Overview
5
+ BERT (Bidirectional Encoder Representations from Transformers) is a set of language models published by Google. They are intended for classification and embedding of text, not for text-generation. See the model card below for benchmarks, data sources, and intended use cases.
6
+
7
+ Weights and Keras model code are released under the [Apache 2 License](https://github.com/keras-team/keras-hub/blob/master/LICENSE).
8
+
9
+ ## Links
10
+
11
+ * [Bert Quickstart Notebook](https://www.kaggle.com/code/matthewdwatson/bert-quickstart)
12
+ * [Bert API Documentation](https://keras.io/api/keras_hub/models/bert/)
13
+ * [Bert Model Card](https://github.com/google-research/bert/blob/master/README.md)
14
+ * [KerasHub Beginner Guide](https://keras.io/guides/keras_hub/getting_started/)
15
+ * [KerasHub Model Publishing Guide](https://keras.io/guides/keras_hub/upload/)
16
+
17
+ ## Installation
18
+
19
+ Keras and KerasHub can be installed with:
20
+
21
+ ```
22
+ pip install -U -q keras-hub
23
+ pip install -U -q keras>=3
24
+ ```
25
+
26
+ Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instruction on installing them in another environment see the [Keras Getting Started](https://keras.io/getting_started/) page.
27
+
28
+ ## Presets
29
+
30
+ The following model checkpoints are provided by the Keras team. Full code examples for each are available below.
31
+
32
+ | Preset name | Parameters | Description |
33
+ |------------------------|------------|-------------------------------------------------------------------------------------------------|
34
+ | `bert_tiny_en_uncased` | 4.39M | 2-layer BERT model where all input is lowercased. |
35
+ | `bert_small_en_uncased` | 28.76M | 4-layer BERT model where all input is lowercased. |
36
+ | `bert_medium_en_uncased` | 41.37M | 8-layer BERT model where all input is lowercased. |
37
+ | `bert_base_en_uncased` | 109.48M | 12-layer BERT model where all input is lowercased. |
38
+ | `bert_base_en` | 108.31M | 12-layer BERT model where case is maintained. |
39
+ | `bert_base_zh` | 102.27M | 12-layer BERT model. Trained on Chinese Wikipedia. |
40
+ | `bert_base_multi` | 177.85M | 12-layer BERT model where case is maintained. |
41
+ | `bert_large_en_uncased` | 335.14M | 24-layer BERT model where all input is lowercased. |
42
+ | `bert_large_en` | 333.58M | 24-layer BERT model where case is maintained. |
43
+
44
+ ### Example Usage
45
+ ```python
46
+ import keras
47
+ import keras_hub
48
+ import numpy as np
49
+ ```
50
+
51
+ Raw string data.
52
+ ```python
53
+ features = ["The quick brown fox jumped.", "I forgot my homework."]
54
+ labels = [0, 3]
55
+
56
+ # Pretrained classifier.
57
+ classifier = keras_hub.models.BertClassifier.from_preset(
58
+ "bert_large_en_uncased",
59
+ num_classes=4,
60
+ )
61
+ classifier.fit(x=features, y=labels, batch_size=2)
62
+ classifier.predict(x=features, batch_size=2)
63
+
64
+ # Re-compile (e.g., with a new learning rate).
65
+ classifier.compile(
66
+ loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
67
+ optimizer=keras.optimizers.Adam(5e-5),
68
+ jit_compile=True,
69
+ )
70
+ # Access backbone programmatically (e.g., to change `trainable`).
71
+ classifier.backbone.trainable = False
72
+ # Fit again.
73
+ classifier.fit(x=features, y=labels, batch_size=2)
74
+ ```
75
+
76
+ Preprocessed integer data.
77
+ ```python
78
+ features = {
79
+ "token_ids": np.ones(shape=(2, 12), dtype="int32"),
80
+ "segment_ids": np.array([[0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0]] * 2),
81
+ "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]] * 2),
82
+ }
83
+ labels = [0, 3]
84
+
85
+ # Pretrained classifier without preprocessing.
86
+ classifier = keras_hub.models.BertClassifier.from_preset(
87
+ "bert_large_en_uncased",
88
+ num_classes=4,
89
+ preprocessor=None,
90
+ )
91
+ classifier.fit(x=features, y=labels, batch_size=2)
92
+ ```
93
+
94
+ ## Example Usage with Hugging Face URI
95
+
96
+ ```python
97
+ import keras
98
+ import keras_hub
99
+ import numpy as np
100
+ ```
101
+
102
+ Raw string data.
103
+ ```python
104
+ features = ["The quick brown fox jumped.", "I forgot my homework."]
105
+ labels = [0, 3]
106
+
107
+ # Pretrained classifier.
108
+ classifier = keras_hub.models.BertClassifier.from_preset(
109
+ "hf://keras/bert_large_en_uncased",
110
+ num_classes=4,
111
+ )
112
+ classifier.fit(x=features, y=labels, batch_size=2)
113
+ classifier.predict(x=features, batch_size=2)
114
+
115
+ # Re-compile (e.g., with a new learning rate).
116
+ classifier.compile(
117
+ loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
118
+ optimizer=keras.optimizers.Adam(5e-5),
119
+ jit_compile=True,
120
+ )
121
+ # Access backbone programmatically (e.g., to change `trainable`).
122
+ classifier.backbone.trainable = False
123
+ # Fit again.
124
+ classifier.fit(x=features, y=labels, batch_size=2)
125
+ ```
126
+
127
+ Preprocessed integer data.
128
+ ```python
129
+ features = {
130
+ "token_ids": np.ones(shape=(2, 12), dtype="int32"),
131
+ "segment_ids": np.array([[0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0]] * 2),
132
+ "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]] * 2),
133
+ }
134
+ labels = [0, 3]
135
+
136
+ # Pretrained classifier without preprocessing.
137
+ classifier = keras_hub.models.BertClassifier.from_preset(
138
+ "hf://keras/bert_large_en_uncased",
139
+ num_classes=4,
140
+ preprocessor=None,
141
+ )
142
+ classifier.fit(x=features, y=labels, batch_size=2)
143
+ ```