28.md

Results of GPT-3.5 on HumanEval/28

Date: 18 July, 2023
Model: gpt-3.5-turbo
Temperature: 0.2
Passing runs: 10
Total runs: 10
Success rate: 100%

Prompt

Tokens: 83

from typing import List


def concatenate(strings: List[str]) -> str:
    """ Concatenate list of strings into a single string
    >>> concatenate([])
    ''
    >>> concatenate(['a', 'b', 'c'])
    'abc'
    """

The above is an incomplete Python code fragment. Return the complete and correct code with no additional text.

Correct solution

from typing import List


def concatenate(strings: List[str]) -> str:
    """ Concatenate list of strings into a single string
    >>> concatenate([])
    ''
    >>> concatenate(['a', 'b', 'c'])
    'abc'
    """return ''.join(strings)

Unit tests

METADATA = {
    'author': 'jt',
    'dataset': 'test'
}


def check(candidate):
    assert candidate([]) == ''
    assert candidate(['x', 'y', 'z']) == 'xyz'
    assert candidate(['x', 'y', 'z', 'w', 'k']) == 'xyzwk'

Runs

✅ Run 1

Start time: 2023-07-18 17:25:11
Duration: 2.95s
Completion tokens: 57
Result: Pass

from typing import List


def concatenate(strings: List[str]) -> str:
    """ Concatenate list of strings into a single string
    >>> concatenate([])
    ''
    >>> concatenate(['a', 'b', 'c'])
    'abc'
    """
    return ''.join(strings)

✅ Run 2

Start time: 2023-07-18 17:25:09
Duration: 2.24s
Completion tokens: 57
Result: Pass

from typing import List


def concatenate(strings: List[str]) -> str:
    """ Concatenate list of strings into a single string
    >>> concatenate([])
    ''
    >>> concatenate(['a', 'b', 'c'])
    'abc'
    """
    return ''.join(strings)

✅ Run 3

Start time: 2023-07-18 17:25:06
Duration: 2.87s
Completion tokens: 57
Result: Pass

from typing import List


def concatenate(strings: List[str]) -> str:
    """ Concatenate list of strings into a single string
    >>> concatenate([])
    ''
    >>> concatenate(['a', 'b', 'c'])
    'abc'
    """
    return ''.join(strings)

✅ Run 4

Start time: 2023-07-18 17:25:03
Duration: 2.54s
Completion tokens: 57
Result: Pass

from typing import List


def concatenate(strings: List[str]) -> str:
    """ Concatenate list of strings into a single string
    >>> concatenate([])
    ''
    >>> concatenate(['a', 'b', 'c'])
    'abc'
    """
    return ''.join(strings)

✅ Run 5

Start time: 2023-07-18 17:25:00
Duration: 2.8s
Completion tokens: 57
Result: Pass

from typing import List


def concatenate(strings: List[str]) -> str:
    """ Concatenate list of strings into a single string
    >>> concatenate([])
    ''
    >>> concatenate(['a', 'b', 'c'])
    'abc'
    """
    return ''.join(strings)

✅ Run 6

Start time: 2023-07-18 17:24:58
Duration: 2.6s
Completion tokens: 57
Result: Pass

from typing import List


def concatenate(strings: List[str]) -> str:
    """ Concatenate list of strings into a single string
    >>> concatenate([])
    ''
    >>> concatenate(['a', 'b', 'c'])
    'abc'
    """
    return ''.join(strings)

✅ Run 7

Start time: 2023-07-18 17:24:55
Duration: 2.82s
Completion tokens: 57
Result: Pass

from typing import List


def concatenate(strings: List[str]) -> str:
    """ Concatenate list of strings into a single string
    >>> concatenate([])
    ''
    >>> concatenate(['a', 'b', 'c'])
    'abc'
    """
    return ''.join(strings)

✅ Run 8

Start time: 2023-07-18 17:24:52
Duration: 2.56s
Completion tokens: 57
Result: Pass

from typing import List


def concatenate(strings: List[str]) -> str:
    """ Concatenate list of strings into a single string
    >>> concatenate([])
    ''
    >>> concatenate(['a', 'b', 'c'])
    'abc'
    """
    return ''.join(strings)

✅ Run 9

Start time: 2023-07-18 17:24:49
Duration: 2.77s
Completion tokens: 57
Result: Pass

from typing import List


def concatenate(strings: List[str]) -> str:
    """ Concatenate list of strings into a single string
    >>> concatenate([])
    ''
    >>> concatenate(['a', 'b', 'c'])
    'abc'
    """
    return ''.join(strings)

✅ Run 10

Start time: 2023-07-18 17:24:47
Duration: 2.45s
Completion tokens: 57
Result: Pass

from typing import List


def concatenate(strings: List[str]) -> str:
    """ Concatenate list of strings into a single string
    >>> concatenate([])
    ''
    >>> concatenate(['a', 'b', 'c'])
    'abc'
    """
    return ''.join(strings)