I tested chatGPT to ask for quick coding help
List of contend
- Introduction
- Trying ChatGPT for Coding Questions
- Observation: Previous Answers Influence Following Answers
- Conclusion
Introduction
Just a heads-up; If you have used chatGBT or GBT3 for coding, you might want to just skim or skip this article.
Beginning of December `22, OpenAI released ChatGPT. A powerful neural Network, based on the GenerativePretrainedTransfomer autoregressive language engine. Transformers have been used in generative language for “some time” now. However, one problem was always, that GPTs trained on massive amounts of text from the internet tend to result in toxic/hatespeech/fakenews generators, due to the nature of internet communication culture and the problem to manually control the large input training sets. Still, ChatGPT uses humans in parts of the training process. First, human annotators seed desired outputs and later rank the output during training, which is then used again in a reward based model. Based on their paper, this leads to reduced toxicity, false information, and an overall better chat.
So far the main use for GPT model I recognized was mainly for creating text in a text scenario like with the JARVIS tool generating blogposts, answering online reviews, generating marketing texts, rewriting your text, or simply chatbots.
Me as a programmer, I had my own first real stunning moment when GitHub Copilot was introduced.
Now, these days, I work on clients laptops, without access to the magic of GitHub Copilot. So I am back to the old “all by yourselves and stackoverflow” phase.
When recently starting with PySpark, I had a lot of basic questions which all can be answered with a quick google search, however, you always have to skim to the right link and find the answer, going through the introductions (just like here ;-) ) and finally not find the answer.
Trying ChatGPT for Coding Questions
So when having another simple PySpark question, I tried ChatGPT for fun and … cool. It gave me great results in a fast and clean manner.
how can I use pyspark where command to search a string wildcard
Here we get an on-point answer. No long introductions, but simply an answer. And also in always the same format.
Now, let us see if it can handle specific questions (of course it can) or if it just use Stackoverflow to extract upvoted answers.
First an older Question on Stackoverflow with 40 upvotes and the answer has 74. So, not in the thousands, but it should weigh:
Tensorboard not found as magic function in jupyter
The answer on stackoverflow:
So interestingly, it did not mention the %load_ext command. The answer is still correct and also with a nice explanation, but it seems not to have read Vlads answer. But is that true?
So it is intended as a chat, so let's continue the chat. Knowing Vlads answer and its regard to the TensorFlow versions, let's specify the question:
how about Tensorflow version newer than tf2.0
Now, the answer is actually a combination of Vlads answer and his update.
Of course the answer is not exactly Vlads, as GPT is not a text scraping algorithm, but it understood the meaning of the question and answer. Also, the answer will most likely be in this form somewhere on some TensorFlow website. Nevertheless, it is nice to see that the correct answer can be found by chatting a bit with chatGPT.
Observation: Previous Answers Influence Following Answers
One thing I noticed is that chatGPT does include its previous answer, altering the outcome of his answers. If this is a limitation is not for me to determine.
But when I asked :
What algorithm is best to identify Bitcoin price development?
looking for something like an LSTM answer, chatGPT answered correctly:
Otherwise, we would all be millionaires and the latest drop in Crypto would not have come as a surprise. However, when asking a follow-up question
What algorithm is best to identify house price development?
which there are clearly some algorithms, shown in every tutorial for regression, random forest, etc., the answer was still:
The answer is still correct, but it based it answer on the Bitcoin answer, which both state: too many variables make it “impossible” to predict it.
While asking the same question without the previous Bitcoin question, the answer is more informative in my opinion:
Here it does state that linear regression etc., can be used, pointing the questioner in the right direction of specific algorithms.
p.s. Asking the same question twice, will give you the same answer.
Is the influence of previous questions good or bad?
Not my place to judge. But generally, this is intended as a real live chat, where previous answers do steer a conversation. So, I guess it works as intended, but you might have to discuss with it sometimes to get to the point…just like in real life.
Conclusion
Will I never us Stackoverflow again? Of course not. At some point, you need Stackoverflow for the real detailed answers on your very very specific problem.
In addition, for me Stackoverflow is often the rubber duck, leading to a solution just by shaping your question.
However, how many time does my lazy brain forget how to properly load that specific column with that other specific filter… tired of going through the Google results? -> chatGPT is the answer.
With the knowledge, that it does answer differently and includes the previous conversation, it is still not the know-all-godlike-thing. Sometimes you still have to ask twice to get to the answer, or maybe never get it. Just like in daily life.
p.s. just that you know, you can also give chatGPT a code block, and it will describe to you what the code is doing ;-) And not only that. If you ask it nicely for a flowchart, you will get a nice overview drawn.
