Python clone detection

This is a codebert model for detecting Python clone codes, fine-tuned on the dataset shared by PoolC on Hugging Face Hub. The original source code for using the model can be found at https://github.com/sangHa0411/CloneDetection/blob/main/inference.py.

How to use

To use the model in an efficient way, you can refer to this repository: https://github.com/RepoAnalysis/PythonCloneDetection, which contains a class that integrates data preprocessing, input tokenization, and model inferencing.

You can also follow the original inference source code at https://github.com/sangHa0411/CloneDetection/blob/main/inference.py.

More conveniently, a pipeline for this model has been implemented, and you can initialize it with only two lines of code:

from transformers import pipeline

pipe = pipeline(model="Lazyhope/python-clone-detection", trust_remote_code=True)

To use it, pass a tuple of code pairs:

code1 = """def token_to_inputs(feature):
    inputs = {}
    for k, v in feature.items():
        inputs[k] = torch.tensor(v).unsqueeze(0)

    return inputs"""
code2 = """def f(feature):
    return {k: torch.tensor(v).unsqueeze(0) for k, v in feature.items()}"""

is_clone = pipe((code1, code2))
is_clone
# {False: 1.3705984201806132e-05, True: 0.9999862909317017}

Credits

We would like to thank the original team and authors of the model and the fine-tuning dataset:

Lincese

This model is released under the MIT license.