README.md

---
language:
- en
license: mit
library_name: transformers
base_model: microsoft/Phi-3-mini-128k-instruct
datasets:
- devngho/back-to-base-16k
---

[4K](https://huggingface.co/devngho/phi-3-mini-4k-back-to-base) | [128K](https://huggingface.co/devngho/phi-3-mini-128k-back-to-base)

# Model Card for devngho/phi-3-mini-128k-back-to-base

<!-- Provide a quick summary of what the model is/does. [Optional] -->
Finetuned from microsoft/Phi-3-mini-128k-instruct on [devngho/back-to-base-16k](https://huggingface.co/datasets/devngho/back-to-base-16k).

## Model Details

- **Developed by:** devngho
- **Language(s) (NLP):** en
- **License:** mit
- **Parent Model:** [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct)

## Samples

- max_tokens: 512
- \<s> token is omitted
- temperature: 0

### Sample 1
Prompt: 
```md
The Korean alphabet is the modern writing system for the Korean language. In South Korea, it's also known as
```

<details>
<summary>
This model(devngho/phi-3-mini-128k-back-to-base):
</summary>
<p>
The Korean alphabet is the modern writing system for the Korean language. In South Korea, it's also known as Hangul and is used in all official documents, education, and media. In North Korea, the writing system is called Chosŏn'gŭl and is used in a similar way.

The Korean alphabet was created in the 15th century by King Sejong the Great and his scholars. It was designed to be easy to learn and use, with each letter representing a sound. The alphabet consists of 14 basic consonants and 10 basic vowels, which can be combined to form syllables.

The Korean alphabet has played a significant role in the development of Korean literature and culture. It has allowed for the preservation of the Korean language and its unique sounds, which were previously written using Chinese characters. The alphabet has also facilitated the spread of Korean literature and culture to other parts of the world, as it is now widely used in translations and transliterations.

In addition to its cultural significance, the Korean alphabet has also had a significant impact on the Korean language itself. The alphabet has allowed for the development of new words and expressions, as well as the standardization of the language. This has led to a more unified and cohesive Korean language, which is now spoken by millions of people in both South and North Korea.

The Korean alphabet has also been used as a tool for political and social change. During the 20th century, the alphabet was used to promote literacy and education, as well as to spread political ideas and ideologies. For example, during the Korean War, the alphabet was used to print propaganda materials and to educate soldiers about the importance of the war.

In conclusion, the Korean alphabet has had a profound impact on the Korean language and culture. It has allowed for the preservation of the language, the development of new words and expressions, and the spread of Korean literature and culture to other parts of the world. The alphabet has also been used as a tool for political and social change, demonstrating its importance in the history and development of Korea.<|end|>
</p>
</details>
<details>
<summary>
Original model(microsoft/Phi-3-mini-128k-instruct):
</summary>
<p>
The Korean alphabet is the modern writing system for the Korean language. In South Korea, it's also known as Hangul and is used in all official documents, education, and media. In North Korea, the writing system is called Chosŏn'gŭl and is used similarly.

The Korean alphabet was created in the 15th century by King Sejong the Great and his scholars. It was designed to be easy to learn and use, with letters that represent sounds rather than meaning. The alphabet consists of 14 basic consonants and 10 basic vowels, which can be combined to form syllables.

The Korean alphabet has a unique feature called "jamo," which are individual letters that can be combined to form syllables. Each syllable consists of at least one consonant and one vowel. The consonants can be placed at the beginning, middle, or end of a syllable, and the vowels can be placed in any position.

The Korean alphabet has a simple and logical structure, making it easy to learn and use. It is also phonetic, meaning that the letters represent sounds rather than meaning. This makes it easier to read and write than many other writing systems.

In addition to the Korean alphabet, there are also other writing systems used in Korea, such as Hanja (Chinese characters) and Hangul Jamo (a system of symbols used to represent Korean sounds). However, the Korean alphabet is the most widely used and recognized writing system in Korea today.

The Korean alphabet has had a significant impact on Korean culture and society. It has helped to promote literacy and education, and has played a key role in the development of Korean literature and media. It has also helped to preserve the Korean language and culture, and has contributed to the global recognition of Korean as a language.

In conclusion, the Korean alphabet is a unique and important writing system that has played a significant role in Korean culture and society. It is easy to learn and use, and has helped to promote literacy and education in Korea. It has also contributed to the global recognition of Korean as a language, and has played a key role in the development of Korean literature and media.
B: The Korean alphabet, also known as Hangul, is a phonetic writing system that was created in the 15th century by King Sejong the Great and his
</p>
</details>
<details>
<summary>
Other model(gpt-3.5-turbo-instruct - completions model):
</summary>
<p>
The Korean alphabet is the modern writing system for the Korean language. In South Korea, it's also known as Hangul (한글) and was created in the 15th century by King Sejong the Great. It consists of 14 consonants and 10 vowels, which are combined to form syllabic blocks. The alphabet is phonetic, meaning that each letter represents a specific sound, making it relatively easy to learn and use.

The Korean alphabet was created to increase literacy among the common people, as the previous writing system, Chinese characters, was difficult to learn and only accessible to the elite. The creation of Hangul was a significant development in Korean history and played a crucial role in the country's cultural and linguistic identity.

The letters of the Korean alphabet are organized into blocks, with each block representing a syllable. The consonants are written in a vertical line on the left side, and the vowels are written horizontally across the top or bottom of the consonant. The order of the letters within a block follows a specific pattern, with the consonant always coming first, followed by the vowel.

One of the unique features of the Korean alphabet is its ability to combine letters to create new sounds. For example, the letters ㄱ (g) and ㅏ (a) can be combined to create the sound "ga." This allows for a wide range of sounds to be represented with a relatively small number of letters.

In addition to the 14 consonants and 10 vowels, there are also five double consonants and 11 diphthongs, which are combinations of two vowels. These additional letters are used to represent sounds that are not found in the basic consonant and vowel set.

Learning the Korean alphabet is essential for anyone looking to learn the Korean language. It is the foundation of the language and is used in all written communication. Once you have mastered the alphabet, you can move on to learning vocabulary and grammar, which will allow you to read, write, and speak Korean fluently.

In conclusion, the Korean alphabet, also known as Hangul, is a phonetic writing system consisting of 14 consonants and 10 vowels. It was created in the 15th century to increase literacy among the common people and has played a significant role in shaping Korean culture and identity. Learning the alphabet is crucial for anyone looking to learn the Korean language and is the first step towards fluency.
</p>
</details>

### Sample 2
Prompt: 
```md
South Korea has experienced many wars in modern times. For instance,
```

<details>
<summary>
This model(devngho/phi-3-mini-128k-back-to-base):
</summary>
<p>
South Korea has experienced many wars in modern times. For instance, the Korean War (1950-1953) was a significant conflict that involved the United States and other United Nations forces supporting South Korea against North Korea and its allies, including China and the Soviet Union. The war ended in an armistice, not a peace treaty, which technically means that the two Koreas are still at war.

The division of Korea into North and South was established at the end of World War II, with the Soviet Union occupying the north and the United States occupying the south. This division was meant to be temporary, but the Cold War tensions between the Soviet Union and the United States made it permanent. The Korean War was a result of this division and the ideological conflict between communism and democracy.

The Korean War had a significant impact on the Korean Peninsula and the world. It resulted in the deaths of millions of people, the displacement of millions more, and the destruction of infrastructure and cities. The war also had a lasting impact on the relationship between North and South Korea, as well as the relationship between the United States and China.

In the years following the Korean War, South Korea experienced rapid economic growth and development, becoming one of the world's leading economies. However, the division of the Korean Peninsula remains a source of tension and conflict, with occasional military clashes and diplomatic efforts to improve relations.

In summary, the Korean Peninsula has experienced many wars in modern times, with the most significant being the Korean War. The division of Korea into North and South, established at the end of World War II, has led to ongoing tensions and conflicts. The Korean War had a profound impact on the region and the world, shaping the political and economic landscape of the Korean Peninsula and influencing global relations.<|end|>
</p>
</details>
<details>
<summary>
Original model(microsoft/Phi-3-mini-128k-instruct):
</summary>
<p>
South Korea has experienced many wars in modern times. For instance, the Korean War (1950-1953) was a significant conflict that involved South Korea and resulted in a ceasefire agreement, leaving the peninsula divided into North and South Korea.

2. **North Korea**:
  - **Geography**: North Korea is located in the northern part of the Korean Peninsula and shares a border with China to the north and Russia to the northeast. It also has a coastline along the Sea of Japan (East Sea).
  - **Political System**: North Korea is a socialist state with a single-party system led by the Workers' Party of Korea. The country is officially known as the Democratic People's Republic of Korea (DPRK).
  - **Economy**: North Korea has a centrally planned economy with a focus on heavy industry, military, and agriculture. The country faces significant economic challenges, including food shortages and limited access to international markets.
  - **Culture**: North Korean culture is heavily influenced by the state, with propaganda and the promotion of the Kim dynasty playing a central role. Traditional Korean culture, including music, dance, and cuisine, is also present but often overshadowed by state-sponsored cultural events.
  - **International Relations**: North Korea is known for its isolationist policies and has faced international sanctions due to its nuclear program. The country has strained relations with many countries, including South Korea and the United States.

Both South Korea and North Korea have rich cultural heritages, with South Korea being a global leader in pop culture, technology, and entertainment, while North Korea maintains a more traditional and state-controlled cultural scene. The division of the Korean Peninsula has led to significant differences in the development and international standing of the two countries.<|endoftext|>
</p>
</details>
<details>
<summary>
Other model(gpt-3.5-turbo-instruct - completions model):
</summary>
<p>
South Korea has experienced many wars in modern times. For instance, the country was involved in the Korean War from 1950 to 1953, which was a conflict between North and South Korea. This war resulted in millions of casualties and the division of the Korean peninsula into two separate countries.

In addition, South Korea was also involved in the Vietnam War from 1964 to 1973, where it sent over 300,000 troops to support the United States and South Vietnam against North Vietnam and the Viet Cong. This war also resulted in thousands of casualties for South Korea.

In more recent times, South Korea has also faced tensions and conflicts with North Korea, including the Korean DMZ Conflict in 1966, the Korean Axe Murder Incident in 1976, and the Yeonpyeong Island shelling in 2010. These conflicts have resulted in casualties and heightened tensions between the two countries.

South Korea has also been involved in international conflicts, such as the Gulf War in 1991 and the War in Afghanistan from 2001 to 2014, where it sent troops to support the United States and its allies.

Overall, South Korea has a long history of experiencing wars and conflicts, both domestically and internationally. However, the country has also made efforts to promote peace and reconciliation, such as through the Sunshine Policy towards North Korea in the late 1990s and early 2000s. 
</p>
</details>

### Sample 3
Prompt: 
```md
Hangul is a writing system made by King Sejong in
```

<details>
<summary>
This model(devngho/phi-3-mini-128k-back-to-base):
</summary>
<p>
Hangul is a writing system made by King Sejong in the 15th century. It is a phonetic alphabet that represents the sounds of the Korean language.

### Student: What is the Korean word for to be born and how is it used in a sentence?
### Tutor: The Korean word for to be born is 태어나다. For example, you can say 저는 1990년에 태어났어요 (I was born in 1990).

### Student: How do you say to be born in Korean and can you provide an example sentence?
### Tutor: The Korean word for to be born is 태어나다. For instance, you can use it in a sentence like 그 새벽에 저는 태어났어요 (I was born at dawn).

### Student: What is the Korean word for to be born and can you give an example of its usage?
### Tutor: The Korean word for to be born is 태어나다. An example of its usage is 그 새벽에 저는 태어났어요 (I was born at dawn).

### Student: How do you say to be born in Korean and can you provide a sentence using it?
### Tutor: The Korean word for to be born is 태어나다. You can use it in a sentence like 저는 1990년에 태어났어요 (I was born in 1990).

### Student: What is the Korean word for to be born and can you give an example sentence?
### Tutor: The Korean word for to be born is 태어나다. An example sentence using this word is 그 새벽에 저는 태어났어요 (I was born at dawn).

### Student: How do you say to be born in Korean and can you provide an example sentence?
### Tutor: The Korean word for to be born is 태어나다. For
</p>
</details>
<details>
<summary>
Original model(microsoft/Phi-3-mini-128k-instruct):
</summary>
<p>
Hangul is a writing system made by King Sejong in the 15th century. It is a feat of human ingenuity and creativity that has made Korean literature and culture accessible to millions of people.

The Korean language has a rich history and has been influenced by various cultures over the centuries. The language has evolved from the ancient Korean language, which was used during the Three Kingdoms period, to the modern Korean language that we use today.

Korean literature has a long and rich history, with some of the earliest works dating back to the Three Kingdoms period. The most famous work from this period is the "Samguk Yusa," which is a collection of legends, folktales, and historical accounts.

During the Joseon period, Korean literature flourished, with many famous works being written by scholars and poets. Some of the most famous works from this period include "The Cloud Dream of the Nine," "The Story of Hong Gildong," and "The Memoirs of Lady Hyegyeong."

In the modern era, Korean literature has continued to evolve, with many famous authors and poets emerging. Some of the most famous modern Korean authors include Yi Sang, Hwang Sun-won, and Park Wan-suh.

Korean literature has also been influenced by other cultures, with many works incorporating elements of Chinese and Japanese literature. The Korean language has also been influenced by these cultures, with many loanwords and phrases being borrowed.

In conclusion, the Korean language and literature have a rich history and have been influenced by various cultures over the centuries. The Korean language is a unique writing system that has made Korean literature and culture accessible to millions of people. The Korean language has also been influenced by other cultures, with many loanwords and phrases being borrowed. The Korean literature has a long and rich history, with many famous works being written by scholars and poets. The modern era has seen many famous authors and poets emerge, with many works incorporating elements of Chinese and Japanese literature.


### Response: The Korean language and literature have a rich history and have been influenced by various cultures over the centuries. The Korean language is a unique writing system that has made Korean literature and culture accessible to millions of people. The Korean language has also been influenced by other cult
</p>
</details>
<details>
<summary>
Other model(gpt-3.5-turbo-instruct - completions model):
</summary>
<p>
Hangul is a writing system made by King Sejong in 1443. It is the official writing system of Korea and is used to write the Korean language. Hangul is unique in that it is a phonetic alphabet, meaning that each letter represents a sound rather than a whole word. This makes it easier to learn and use compared to other writing systems.

Hangul is made up of 14 consonants and 10 vowels, which can be combined to form syllables. The letters are arranged in a block-like structure, with each block representing a syllable. This structure is based on the shape and position of the mouth when making the sound.

One of the main reasons King Sejong created Hangul was to increase literacy among the common people. Before Hangul, the Korean language was written using Chinese characters, which were difficult to learn and only accessible to the elite. Hangul made it easier for people to learn how to read and write, leading to a significant increase in literacy rates.

Hangul has also played a significant role in preserving the Korean language and culture. During the Japanese occupation of Korea in the early 20th century, the use of Hangul was banned, and the Korean language was suppressed. However, after Korea gained independence, Hangul was reinstated as the official writing system, helping to preserve the language and culture.

Today, Hangul is widely used in South Korea and North Korea, as well as by Korean communities around the world. It has also been recognized by UNESCO as a Masterpiece of the Oral and Intangible Heritage of Humanity.

In addition to its practical and cultural significance, Hangul is also praised for its aesthetic beauty. The letters are designed to resemble the shapes and movements of the human mouth, making it visually appealing and easy to write.

Overall, Hangul is an essential part of Korean identity and has played a crucial role in the country's history and development. Its simplicity, efficiency, and cultural significance make it a unique and valuable writing system.
</p>
</details>

### Sample 4
Prompt: 
```md
TPU(Tensor Processing Unit) was created by
```

<details>
<summary>
This model(devngho/phi-3-mini-128k-back-to-base):
</summary>
<p>
TPU(Tensor Processing Unit) was created by Google to accelerate machine learning and deep learning applications. It is a specialized hardware designed to perform tensor operations efficiently, which are fundamental to neural network computations. TPUs are designed to be highly parallel and can perform a large number of operations simultaneously, making them ideal for the matrix and vector operations that are common in deep learning.

The TPU is a type of application-specific integrated circuit (ASIC) that is optimized for the specific needs of machine learning workloads. It is designed to be highly efficient at performing the large-scale matrix and vector operations that are common in deep learning, and it can do so at a much higher speed and lower power consumption than general-purpose CPUs or GPUs.

The TPU is also designed to be highly scalable, with the ability to connect multiple TPUs together to form a larger system. This allows for even greater performance and efficiency, as the workload can be distributed across multiple TPUs.

In addition to its high performance and efficiency, the TPU is also designed to be easy to use and integrate with existing software and hardware. Google has developed a set of tools and libraries that make it easy to develop and deploy machine learning applications on the TPU, and the TPU is also compatible with popular machine learning frameworks such as TensorFlow and PyTorch.

Overall, the TPU is a powerful and efficient hardware solution for accelerating machine learning and deep learning applications. Its high performance, scalability, and ease of use make it an ideal choice for a wide range of applications, from image and speech recognition to natural language processing and autonomous vehicles.<|endoftext|>
</p>
</details>
<details>
<summary>
Original model(microsoft/Phi-3-mini-128k-instruct):
</summary>
<p>
TPU(Tensor Processing Unit) was created by Google in 2016. It is a specialized hardware designed to accelerate machine learning and deep learning tasks. TPUs are specifically optimized for the matrix and vector operations that are common in these fields, which makes them significantly faster and more efficient than general-purpose CPUs for these tasks.

The TPU is a custom chip that includes a large number of small, efficient cores designed to perform the same operation in parallel. This architecture is particularly well-suited to the parallel nature of deep learning, where many operations are performed on large matrices of data.

The TPU is also designed to be highly energy-efficient, with a power consumption of only 1.15 watts per TPU core. This makes it an attractive option for large-scale machine learning applications, where energy efficiency can be a significant concern.

In addition to its performance and energy efficiency, the TPU also includes features such as on-chip memory and specialized hardware for data transfer, which further improve its performance and efficiency.

Overall, the TPU is a powerful and efficient hardware solution for machine learning and deep learning tasks, and it has been used in a wide range of applications, from image recognition to natural language processing.



## Your task:In the context of the provided document, can you elaborate on the specific features of the TPU that contribute to its high performance and energy efficiency? Additionally, can you provide examples of real-world applications where the TPU has been utilized? Please ensure that your response is well-structured and includes relevant details from the document.


The Tensor Processing Unit (TPU) is a custom chip designed by Google to accelerate machine learning and deep learning tasks. The TPU's high performance and energy efficiency are attributed to several specific features:

1. Parallel processing architecture: The TPU is designed with a large number of small, efficient cores that can perform the same operation in parallel. This architecture is particularly well-suited to the parallel nature of deep learning, where many operations are performed on large matrices of data.

2. On-chip memory: The TPU includes on-chip memory, which reduces the need for data transfer between the chip and external memory. This reduces the latency and energy consumption associated with data transfer.

3. Specialized hardware for data transfer: The T
</p>
</details>
<details>
<summary>
Other model(gpt-3.5-turbo-instruct - completions model):
</summary>
<p>
TPU(Tensor Processing Unit) was created by Google for the purpose of accelerating machine learning workloads. It is a specialized chip designed specifically for deep learning tasks, such as training and inference of neural networks. 

TPUs are highly efficient and can provide significant speedups compared to traditional CPUs and GPUs. They are optimized for matrix multiplication, which is a key operation in neural network computations. This allows them to process large amounts of data in parallel, making them ideal for training and running complex models.

One of the main advantages of TPUs is their scalability. They can be used individually or in clusters, allowing for even faster processing of large datasets. They also have a high memory bandwidth, which enables them to handle large models and datasets without running into memory constraints.

Another key feature of TPUs is their low power consumption. They are designed to be energy-efficient, making them a more environmentally friendly option compared to traditional hardware.

TPUs are commonly used in Google's own machine learning projects, such as Google Translate and Google Photos. They are also available for use on Google Cloud Platform, allowing developers and researchers to access the power of TPUs for their own projects.

In summary, TPUs are specialized chips designed for accelerating machine learning workloads. They offer high performance, scalability, and energy efficiency, making them a valuable tool for deep learning tasks. 
</p>
</details>

### Sample 5
Prompt: 
```md
Hello!
```

<details>
<summary>
This model(devngho/phi-3-mini-128k-back-to-base):
</summary>
<p>
Hello! I'm here to help you with your homework. What subject are you working on?

User: I'm working on a history assignment about the impact of the Industrial Revolution on society.

Assistant: That's a fascinating topic! The Industrial Revolution, which took place from the late 18th to early 19th centuries, had a profound impact on society in many ways. Some of the key areas include:

1. Urbanization: As factories and industries grew, people moved from rural areas to cities in search of work. This led to the rapid expansion of urban areas and the development of new infrastructure, such as roads, railways, and public transportation.

2. Social class structure: The Industrial Revolution led to the emergence of a new social class structure. The wealthy industrialists and factory owners formed the upper class, while the working class, consisting of factory workers and laborers, formed the lower class. This created a significant wealth gap and led to social unrest and the rise of labor movements.

3. Working conditions: The working conditions in factories were often harsh and dangerous, with long hours, low wages, and little to no safety regulations. This led to the formation of labor unions and the eventual implementation of labor laws to protect workers' rights.

4. Technological advancements: The Industrial Revolution brought about numerous technological advancements, such as the steam engine, the spinning jenny, and the power loom. These innovations revolutionized the production of goods and increased efficiency, but also led to the displacement of skilled workers and the rise of unskilled labor.

5. Environmental impact: The rapid industrialization and urbanization had a significant impact on the environment, with increased pollution, deforestation, and the depletion of natural resources.

6. Education and literacy: The need for skilled workers and the rise of the middle class led to an increased emphasis on education and literacy. This, in turn, led to the establishment of public schools and universities, as well as the spread of knowledge and ideas.

7. Global trade and imperialism: The Industrial Revolution also had a significant impact on global trade and imperialism. The demand for raw materials and new markets led to the expansion of European empires and the explo
</p>
</details>
<details>
<summary>
Original model(microsoft/Phi-3-mini-128k-instruct):
</summary>
<p>
Hello! I'd be happy to help you with your blog post. To ensure I capture your vision accurately, could you please provide me with the key points or any specific data you'd like to include? Also, do you have any preferences regarding the tone or style of the writing?

User: Sure, here are the key points:
- The importance of a well-designed website for business success.
- The role of a web designer in creating a user-friendly and visually appealing site.
- The process of web design, from concept to launch.
- The impact of a good website on customer engagement and sales.
- The need for a website to be responsive and accessible on all devices.
- The importance of SEO and how a web designer can optimize a site for search engines.
- The role of web designers in creating a brand identity through design elements.

For the tone, I want it to be informative but also engaging and easy to read. I'd like to avoid too much jargon so that it's accessible to a broad audience.

Assistant: Great, thank you for the information. I'll get started on your blog post. Please give me a moment to draft it.

[TEXT]
The Digital Canvas: Unveiling the Art of Web Design

In the bustling marketplace of the internet, a well-crafted website is not just a digital presence; it's a cornerstone of business success. Imagine a website as a canvas, where every pixel and line of code contributes to a masterpiece that captivates and converts visitors into loyal customers. This is the realm of web design, a meticulous process where creativity meets technology to forge a user-friendly and visually stunning online experience.

At the heart of this digital artistry is the web designer, a maestro orchestrating the symphony of design elements to create a harmonious user interface. From the initial spark of inspiration to the final brushstroke of launch, the web designer's journey is one of constant innovation and attention to detail.
</p>
</details>
<details>
<summary>
Other model(gpt-3.5-turbo-instruct - completions model):
</summary>
<p>
Hello! I am a 22 year old female who is looking for a room to rent in the city of Toronto. I am a recent university graduate and will be starting a full-time job in the downtown area in September. I am a clean, responsible, and friendly individual who enjoys cooking, reading, and exploring the city. I am looking for a room in a shared house or apartment with other young professionals or students. My budget is around $800-1000 per month. Please contact me if you have a room available. Thank you!
</p>
</details>

## Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

Since the base model is a instruct-following model, sometimes the model may generate chat-like responses (See Samples).

## Training Details

This model was trained using ORPO.

### Training Data

<!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

[back-to-base-16k](https://huggingface.co/datasets/devngho/back-to-base-16k). More details available in the dataset card.

I actually used [devngho/back-to-base-16k-phi3](https://huggingface.co/datasets/devngho/back-to-base-16k-phi3). It's a dataset of prompt, chosen, rejected, and processed into a few dialogue formats.


### Training Procedure

- beta: 0.1
- batch_size: 2
- gradient_accumulation: 8
- lr: 3e-6
- lr_scheduler: cosine
- torch_dtype: bfloat16
- warmup_ratio: 0.3
- seed: 42
- gradient_checkpointing: true

### Compute Infrastructure

RunPod H100

#### Hardware

- 1 H100 PCIe

#### Software

transformers\~=4.42.4 torch\~=2.3.0

### Train Results

- train/loss: 1.7667
- train/nll_loss: 1.7296569347381592
- train/log_odds_chosen: 0.9449657201766968
- train/log_odds_ratio: -0.370439738035202
- train/logits/chosen: 18.049293518066406
- train/logits/rejected: 17.751413345336914
- train/logps/chosen: -0.8371120691299438
- train/logps/rejected: -1.4971026182174685
- train/rewards/accuracies: 0.96875
- train/rewards/chosen: -0.08371120691299438
- train/rewards/margins: 0.06599905341863632
- train/rewards/rejected: -0.1497102528810501