update files

README.md

@@ -1,59 +1,27 @@
 ---
-license: other
-base_model: /apdcephfs_cq10/share_919031/larsonwang/LLaMA-Factory/save_model/train_lora_1709305042/
-tags:
-- llama-factory
-- full
-- generated_from_trainer
-model-index:
-- name: train_lora_1709346779
-  results: []
+license: apache-2.0
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
+base_model:https://huggingface.co/google/gemma-2b
 
-# train_lora_1709346779
+the language of model: chinese and english
 
-This model is a fine-tuned version of [/apdcephfs_cq10/share_919031/larsonwang/LLaMA-Factory/save_model/train_lora_1709305042/](https://huggingface.co//apdcephfs_cq10/share_919031/larsonwang/LLaMA-Factory/save_model/train_lora_1709305042/) on the sft_sample dataset.
+The following uses gemma-2b (a language model that only supports English) to train a large model process that supports Chinese and English.
 
-## Model description
+step 1:
+Use SentencePiece(bpe) to train Chinese corpus to obtain tokenizer.model and tokenizer.vocab
 
-More information needed
+step 2：
+Merge the Chinese of tokenizer.model and the original of tokenizer.model
 
-## Intended uses & limitations
+step 3：
+Use the merged special_tokens_map.json, tokenizer.model, tokenizer_config.json to replace the files of the original model (such as gemma-2b)
 
-More information needed
+step 4:
+Use LLaMA-Factory for pre-training. Pay attention to the pre-training parameters. Resize vocab and resize embedding are required.
 
-## Training and evaluation data
+step 5:
+Based on the model pre-trained in step 4, the instructions are fine-tuned, which significantly improves the model's ability to understand and execute instructions.
 
-More information needed
-
-## Training procedure
-
-### Training hyperparameters
-
-The following hyperparameters were used during training:
-- learning_rate: 5e-05
-- train_batch_size: 8
-- eval_batch_size: 8
-- seed: 42
-- distributed_type: multi-GPU
-- num_devices: 8
-- total_train_batch_size: 64
-- total_eval_batch_size: 64
-- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
-- lr_scheduler_type: cosine
-- num_epochs: 5.0
-- mixed_precision_training: Native AMP
-
-### Training results
-
-
-
-### Framework versions
-
-- Transformers 4.38.0
-- Pytorch 2.0.1+cu118
-- Datasets 2.17.1
-- Tokenizers 0.15.2
+step 6:
+Based on the instruction fine-tuning model, we can use this model for SFT training under different specific tasks, so that the model can perform better on specific tasks.