No, ChatGPT is not 10x-ing developer performance

January 25th, 2023
ChatGPT and similar language models have recently been gaining attention for their potential to revolutionise code generation and enhance developer productivity. I was curious to see what all the hype was about, so I decided to try it out for some development work.
Bored robot

After conducting my own research, it became clear that ChatGPT and similar language models are powerful tools that can help developers address various issues. Their ability to solve problems in code, self-optimise and find alternatives to given solutions is particularly impressive. Additionally, ChatGPT can introduce developers to new libraries and tools they may not have been aware of before.

However, it is noteworthy that these tools have severe limitations. The model sometimes generates subtle errors, such as comparisons without conditions or accessing non-existing parts in data structures. Additionally, it may offer pointless improvements or duplicate steps like applying multiple compressions.

Is the speed at which machine learning advances solving these issues soon? Maybe, maybe not. For example, self-driving cars have been just around the corner for many years. Replacing human capabilities is not a linear problem. While car manufacturers have solved most of the problem swiftly, the last bit has been intractable and requires further research and development of an unknown timeframe.

Similarly, ChatGPT code and other generations have matured to impressive abilities quickly. But to make it reliable and trustworthy, the last bit of maturity could be many, many years away. As developers are already exploring the technology, it has the potential to make its way into critical systems such as financial, infrastructure, defence, or healthcare systems. This is why it is important to be cautious and aware of the limitations of these tools.

Developers need to remember to treat output from ChatGPT with caution. While it can be intriguing and helpful, developers should always double-check and adapt the code. This is why colleagues or an interactive community like Stack Overflow, where developers validate, share experiences and discuss the pros and cons of the code, remain superior.

In conclusion, ChatGPT and similar language models are powerful tools that can be valuable for developers. However, they come with limitations which negate many of their benefits. These tools can be a great way to inspire, explore, and learn, but they require experienced developers and good practices to be used safely. They are not tools that will replace developers and make them unnecessary but instead means to help experienced developers do their work more efficiently in the future.

Christian Prokopp, Bold Data, Founder

Note, I used ChatGPT with some bullet points to generate the first draft of this post. That was helpful but the amount of edits needed to get to something worth posting was surprisingly high. This experience feels similar to trying to use it as a code copilot. It looks good on first glance but the devil is in the detail.

    Let's talk

    You have a business problem in need for data and analysis? Send us an email.

    Subscribe to updates

    Join Bold Data's email list to receive free data and updates.

Related Posts

The Power of Schema Enforcement in Delta Lake

Police car light
Prevent errors and inconsistencies with Delta Lake's robust data management technology.

Python TDD with ChatGPT

Being tested
Programming with ChatGPT using an iterative approach is difficult, as I have demonstrated previously. Maybe ChatGPT can benefit from Test-driven development (TDD). Could it aid LLMs as it does humans?

How to code Python with ChatGPT

Meditating robot
Can ChatGPT help you develop software in Python? Let us ask ChatGPT to write code to query AWS Athena to test if and how we can do it step-by-step.

OpenAI GPT-3: Content spam or more?

Robot on a typewriter in a library (DALL·E generate)
OpenAI's ChatGPT has made the news recently as a next-generation conversational agent. It has a surprising breadth which made me wonder, could OpenAI generate specific technology content good enough to post, and what would that imply for the future?

Will Tesla's AI break the insurance market?

Car accident
Insurance works because it shares costs in the face of uncertainty. What happens when Tesla removes uncertainty and distributes cost seemingly more fairly? First partially and eventually wholly? Will insurance fail, doing more harm than good?

Why I became a Solo Founder

Single lego figure walking in sand
I never wanted to be a solo founder. Yet, in 2021, I quit my job and started Bold Data to mine the Internet single-handedly. Trust me, it sounds as insane to write as to read. What on earth possessed me, and more importantly, would I do it again?

All Blog Posts

See the full list of blog posts to read more.
Subscribe for updates, free datasets and analysis.