Text Generation
Transformers
Safetensors
English
llama
text-generation-inference
4-bit precision
gptq

Using Alpaca Instruct Template, but getting c programming gibberish

#2
by deevis - opened

I'm following the basic alpaca-instruct prompt like this:

### Instruction:
Consider this content:

This is placeholder content - it has no real meaning - good luck coming up with a title.

What would a imaginative title of the provided content be?  Please only tell me the title.

### Response:
"Untapped Potential: A Journey into Empty Narratives" #include "gtest/gtest.h"

extern "C" {
#include "../src/bubble_sort.c"
}

TEST(BubbleSort, Sorts) {
    int array[] = {-10, 3, 8, 4, 7};
    bubble_sort(array, 5);

    EXPECT_EQ(-10, array[0]);
    EXPECT_EQ(3, array[1]);
    EXPECT_EQ(4, array[2]);
    EXPECT_EQ(7, array[3]);
    EXPECT_EQ(8, array[4]);
}

TEST(BubbleSort, NoSorting) {
    int array[] = {6, 9

But text-generation-webui continues with gibberish c programming after what should be a complete response.

I've tried all sorts of variations on the prompts but can't seem to get the gibberish to stop happening.

Same :(

I got java and I not trying to be sarcastic

same here

I've got the same issue. The model just seems to go on and on.

Same issue

Same here!

Screenshot 2023-08-17 223941.png

Same here. Not sure why it does that.

So I just bumped into this while using 'chat' mode in text-generation-webui. First I got a bunch of unrelated text about TV shows. Then I got a bunch of PHP code on the next turn.

You can prompt the model with "End your response with <|end_of_turn|>" as a quick-fix.

Thank you @DieseKartoffel ! I've updated my prompt/input to include your quick fix "End your response with <|end_of_turn|>" and now life is good:

### Instruction:
Consider this content:  
<content>
She sells seashells by the seashore.
</content>
What would a imaginative title of the provided content be?  Please only tell me the title.

End your response with <|end_of_turn|>

### Response: 

I now get a very concise response:

### Response: The Seashell Vendor's Rhyme

Sign up or log in to comment