Context is Key: Learnings from a GitHub Copilot Pilot Project
Hung V. Nguyen
GitHub copilot was used in two projects. For the first project, GitHub Copilot generated unit tests and modular code. The second project streamlined coding tasks, facilitating the review and improvement of code quality.
“Deeming this pilot project a success, Infodation will now consider the company wide implementation of GitHub Copilot. ”
The developers found that providing detailed context improves the effectiveness of GitHub Copilot. They summarized their findings as “Context is key”, re-iterating that quality of context significantly boosts the accuracy and usefulness of LLMs suggestions. In other words: Yes, GitHub Copilot builds code based on the lines of script that already in the repository, but it also considers how the code is named, structured, refactored, whether there is cohesion within the code and many other factors. Tweaking this “context” created code tailormade for the individual projects. Here is how our developers did it:
Prompt:
“Explain the function including its inputs, outputs, and the used logic. Highlight key operations or algorithms”
def factorial(n):
result = 1
for i in range(1, n + 1):
result *= i
return result
The factorial function is designed to compute the factorial of a given non-negative integer n. A factorial, denoted as n! is the product of all positive integers less than or equal to n. Here's a detailed breakdown of the function:
Inputs
n: A non-negative integer. This integer represents the number at which the factorial calculation stops.
Outputs
Returns: The factorial of the integer n, calculated as n×(n−1)×(n−2)×...×1n \times (n-1) \times (n-2) \times ... \times 1.
Logic and Key Operations
Initialization: The function starts by initializing a variable result to 1. This initialization is crucial as the factorial of any number starts with the multiplication identity, which is 1.
Iteration: The function uses a for loop to iterate over a range of numbers from 1 to n inclusive. The range function range(1, n + 1) generates a sequence of integers starting at 1 and ending at n, providing the necessary values to compute the factorial.
Multiplication: In each iteration of the loop, the current loop variable ii is multiplied by the current value of the result. The operation result *= i is used, which is shorthand for result = result * i. This operation accumulates the product of all integers from 1 to n.
Return Value: After completing the loop (i.e., once all integers up to n have been multiplied), the function returns the value of the result, which now holds the computed factorial of n.
Key Algorithm
The algorithm employed here is a simple iterative method to compute the factorial of a number. The key operations involve initializing a variable to hold the running product and iteratively multiplying this variable by each number in a specified range.
The time complexity of this function is O(n), where nn is the input number. This is because the function needs to perform nn multiplications — one for each number from 1 to nn. The space complexity is O(1) as the function uses a fixed amount of space regardless of the size of nn.
This implementation is straightforward and efficient for computing factorials of moderately sized numbers. However, for very large numbers, this approach might lead to long computation times and potential integer overflow, depending on the limitations of the programming language's integer type.
def calculate_sum(numbers):
# Initialize sum to zero
total = 0
# Iterate over each number and add to total
for number in numbers:
total += number
return total # Return the computed sum
The function calculate_sum is designed to compute the sum of a list of numbers. Here's a breakdown of its components and operations:
Inputs
numbers: A list of numbers (integers or floats). This is the input to the function, where each element in the list is a number to be summed.
Outputs
Returns: The total sum of all the numbers in the input list.
Logic and Key Operations
Initialization: The function starts by initializing a variable total to zero. This variable is used to store the running total of the numbers as they are added together.
Iteration: The function then enters a loop where it iterates over each element in the input list numbers. For each iteration, the current number is added to total.
Addition: In each iteration of the loop, the += operator is used to add the current number to total. This operator is a shorthand for total = total + number.
Return Value: After the loop has processed all the numbers in the list, the function returns the value of total, which by now holds the sum of all the input numbers.
Key Algorithm
The algorithm used here is a simple iterative method to accumulate a sum. It's a direct approach that operates in O(n) time complexity, where n is the number of elements in the list. This means that the time to compute the sum scales linearly with the number of elements.
Overall, this function is an example of straightforward and efficient use of iteration for summing a sequence of numbers.
The code with comments gives superior results for the example prompt
Prompt:
“Improve the current code”
int[] arr = {1, 2, 3};
int res = 0;
foreach (var item in arr) {
res += item;
}
using System.Linq;
int[] arr = {1, 2, 3};
int res = arr.Sum();int[] numbers = {1, 2, 3};
int total = 0;
foreach (int number in numbers) {
total += number;
}
using System.Linq;
int[] numbers = {1, 2, 3};
int total = numbers.Sum();
Prompt:
“Optimize the FunctionName implementation to reduce its code complexity. Refactor to make it more modular, use clean coding practices, and eliminate redundant or deeply nested logic. Ensure the functionality and readability remain intact”
def calculate_sum(numbers):
# Initialize sum to zero
total = 0
# Iterate over each number and add to total
for number in numbers:
total += number
return total # Return the computed sum
// Function to calculate sum of an array
function calculateSum(array) {
return array.reduce((acc, cur) => acc + cur, 0);
}
// Function to calculate average of an array
function calculateAverage(array) {
if (!Array.isArray(array) || array.length === 0) return null; // Validate input
const sum = calculateSum(array); // Calculate sum
return sum / array.length; // Calculate and return average
}
// Calculates the sum of an array of integers
public int CalculateSum(int[] numbers) {
int sum = 0; // Initialize sum
// Loop through numbers and accumulate the sum
foreach (int num in numbers) {
sum += num; // Add each number to sum
}
return sum; // Return total sum
}
// Function to calculate sum of an array
function calculateSum(array) {
return array.reduce((acc, cur) => acc + cur, 0);
}
// Function to calculate average of an array
function calculateAverage(array) {
if (array.length === 0) return null; // No data to process
let sum = calculateSum(array); // Calculate sum using the helper function
return sum / array.length; // Calculate and return average
}
// Example usage
let array = [10, 20, 30];
let average = calculateAverage(array);
Use case studies or specific project scenarios where these practices improved outcomes
Contextual Prompt:
Suggest code when you are typing comments “Give a managing users module by API in the website”
using System;
using System.Net.Http;
using System.Net.Http.Json;
using System.Threading.Tasks;
public class UserManager
{
private readonly HttpClient _httpClient;
private readonly string _baseUrl;
public UserManager(string baseUrl)
{
_httpClient = new HttpClient();
_baseUrl = baseUrl;
}
public async Task CreateUserAsync(User user)
{
try
{
var response = await _httpClient.PostAsJsonAsync($"{_baseUrl}/users", user);
response.EnsureSuccessStatusCode();
return await response.Content.ReadFromJsonAsync();
}
catch (Exception ex)
{
Console.WriteLine($"Error creating user: {ex.Message}");
return null;
}
}
public async Task GetUserAsync(int userId)
{
try
{
return await _httpClient.GetFromJsonAsync($"{_baseUrl}/users/{userId}");
}
catch (Exception ex)
{
Console.WriteLine($"Error retrieving user: {ex.Message}");
return null;
}
}
public async Task UpdateUserAsync(int userId, User user)
{
try
{
var response = await _httpClient.PutAsJsonAsync($"{_baseUrl}/users/{userId}", user);
response.EnsureSuccessStatusCode();
return await response.Content.ReadFromJsonAsync();
}
catch (Exception ex)
{
Console.WriteLine($"Error updating user: {ex.Message}");
return null;
}
}
public async Task DeleteUserAsync(int userId)
{
try
{
var response = await _httpClient.DeleteAsync($"{_baseUrl}/users/{userId}");
response.EnsureSuccessStatusCode();
return true;
}
catch (Exception ex)
{
Console.WriteLine($"Error deleting user: {ex.Message}");
return false;
}
}
}
public class User
{
public int Id { get; set; }
public string Name { get; set; }
public string Email { get; set; }
}
Prompt:
“Based on the function, write a test case to test”
import requests
from requests.exceptions import RequestException
# Function to make API calls with error handling
def make_api_call(url, method='GET', data=None):
try:
response = requests.request(method, url, data=data)
response.raise_for_status()
return response.json()
except RequestException as e:
return {'error': str(e)}
import unittest
from unittest.mock import patch
from your_module import make_api_call # Replace 'your_module' with the actual name of your module
class TestMakeApiCall(unittest.TestCase):
def test_successful_get_request(self):
# Test a successful GET request
with patch('requests.request') as mocked_get:
mocked_get.return_value.status_code = 200
mocked_get.return_value.json.return_value = {'key': 'value'}
result = make_api_call('http://example.com')
mocked_get.assert_called_once_with('GET', 'http://example.com', data=None)
self.assertEqual(result, {'key': 'value'})
def test_request_raises_exception(self):
# Test handling exceptions like network errors
with patch('requests.request') as mocked_get:
mocked_get.side_effect = RequestException("An error occurred")
result = make_api_call('http://example.com')
mocked_get.assert_called_once_with('GET', 'http://example.com', data=None)
self.assertEqual(result, {'error': 'An error occurred'})
def test_failed_get_request_due_to_status_code(self):
# Test handling responses with bad status codes
with patch('requests.request') as mocked_get:
mocked_get.return_value.raise_for_status.side_effect = requests.exceptions.HTTPError("404 Client Error")
mocked_get.return_value.status_code = 404
result = make_api_call('http://example.com')
mocked_get.assert_called_once_with('GET', 'http://example.com', data=None)
self.assertEqual(result, {'error': '404 Client Error'})
# This allows the test script to be run from the command line
if __name__ == '__main__':
unittest.main()
While there has not been a formal case study regarding the evaluation of the efficiency of GitHub Copilot, anecdotal evidence suggests that its usage improved code quality and accelerates project timelines and was especially fruitful for category testing. Developers were able to focus on creative solutions rather than repetitive coding tasks, resulting in an averaged updated happiness score. It however also became apparent that without a skilled developer interpreting the results, GitHub Copilot lost a lot of its shine. Deeming this pilot project a success, Infodation will now consider the company wide implementation of GitHub Copilot.
