”A staff engineer at an interviewing cloud platform, administered an interview with ChatGPT, he found the tool was able to generate a valid solution. But the way in which it reached its answer was what caught his eye. It behaved more like someone who had memorized the answer to this particular problem in advance but did not have the needed skills to solve it independently, which is consistent with how it works,” he wrote. ChatGPT does not have a model of the problem, only questions, and plausible responses, he said. Moreover, he witnessed “wild changes to its algorithm” and the way that its explanations were misaligned with its actions. He concluded ChatGPT’s results were very unlike a human solving a problem, and it did not handle probing questions in a way that created confidence in its understanding.” Jason Wodicka – Karat
As technology continues to evolve, there’s been increasing interest in the use of artificial intelligence and machine learning to automate various aspects of software development. One area that’s been of particular interest is the use of language models like ChatGPT to assist with coding-related tasks. Some have even speculated that these tools could eventually make coding tests for engineers obsolete. In this blog, we’ll examine this possibility and whether it’s likely to become a reality.
First, let’s consider what coding tests are and why they’re important. Coding tests are typically used as a way to evaluate a developer’s skills and proficiency in a particular programming language or technology. These tests often require candidates to solve real-world problems and demonstrate their ability to write clean, efficient, and maintainable code. They can be used in a variety of contexts, such as during the hiring process or as a way to assess a developer’s ongoing performance.
While coding tests can be time-consuming and challenging, they serve an important purpose in evaluating a developer’s technical skills and problem-solving abilities. They’re also a way to ensure that developers have the skills they need to create high-quality software products that meet business requirements.
Now, let’s consider the potential impact of language models like ChatGPT on coding tests. While ChatGPT can assist with certain aspects of the coding process, such as providing suggestions for syntax and code structure, it’s important to note that these models are not capable of independently solving complex problems and creating sophisticated applications. They also lack the ability to evaluate a developer’s problem-solving and critical thinking abilities, which are key components of coding tests.
Furthermore, while language models can be helpful in assisting with coding-related tasks, they’re not a replacement for human intelligence and expertise. Developers bring unique insights and experiences to their work that can’t be replicated by a machine. Additionally, language models are limited by the quality and quantity of data they’ve been trained on. If they haven’t been exposed to a particular type of problem or technology, they may not be able to provide useful guidance.
While the use of language models like ChatGPT is an exciting development in the field of software development, it’s unlikely that they will make coding tests for engineers obsolete. Coding tests serve a specific purpose in evaluating a developer’s skills and potential, and they can’t be replicated by a machine. Instead, the most likely scenario is that language models will continue to be used as a complementary tool to assist developers in their work, rather than as a replacement for human expertise.