ChatGPT is a bit of a “tongue twister” so I will speak of Cheppy.
AMIS has a long history of spotting, exploring, embracing and rolling out new concepts and technologies . It something we like doing – a trait of the organisation- and are good at and successful with. That means we are not afraid of new things – we welcome them in. But we are critical too: what can it do for us? What is fad or fiction, what is fact and perhaps fate?
Cheppy had been coming for some time – or at least something like Chappy. The large language models that could generate images (StableDiffusion, DALL-E *), MIdjourney, GPT) or the AI powered tools for software developers (GitHub Copilot, AWS Code Whisperer, Google Alphacode, BlueDiff Cover, CodeGeeX) have been around for some time now. We felt close to an inflection point, where capabilities and uptake of these technologies would explode. When Cheppy arrived on the seem, it seemed that point may have been reached. Especially the versatility of Cheppy made it interesting to a wide range of people – from my son to my mother, my tennis team and almost anyone I talk with. (most of the conversations at some point touch on the shortsighted rearguard battles in some universities where Cheppy is tried to be banned)
At AMIS, we started early explorations in what Cheppy could do for us professionally as soon as the service became available. On internal forums we shared our own findings and some of the amazing stories found online (“pretend you are a Linux terminal, now execute these commands”, “explain this story to me as if I were a five year old and do it in language X”). We compared Cheppy to Copilot (we prefer Cheppy), tried to find clues that our own blog (3500 articles) had been a source for Cheppy (it probably was, but no actual proof) and pushed each other for more extravagant and also more useful applications.
In January we organized a Code Café where two colleagues summarized all our findings and discussed the origin of Cheppy and lookalikes, the data sets they were trained on and some history and workings of the large language models. We also looked at the road ahead. And we shared experiences and brainstormed on further applications. Last Tuesday we had another session on Cheppy (well, on ChatGPT but we started calling it Cheppy to be easy on our Dutch native tongues) – looking specifically on the impact on and opportunities for our work in the short to mid term (6 months to one year ahead).
Our assessment:
- Cheppy is a great tool. A power tool that will allow us to be more productive and efficient, perhaps even more creative and produce higher quality work including such deliverables as documentation and tests
- Cheppy can help experienced, skilled software engineers – but it will not enable someone without proper skills and experience
- Cheppy feels like an assistant who can scan and summarize 100s of Google search results almost instantly; for someone who can search and assess such results, this assistant is a marvel and a great help; for someone who cannot currently process that information, Cheppy will probably be of limited help
- Cheppy returns – with great aplomb and apparent conviction- incorrect answers; sometimes the logical extrapolation from correct information about an API or library that for some illogical reason does not actually exist in that API or library – that can highly confuse the human developer trying to apply that so convincingly produced result (that then is meekly apologized for). Verification of incorrect results can take a lot of time and undo some of the productivity gains
- Cheppy does not understand what it is saying. It can not judge whether the information it provides (“guessed”) is correct or whether code is “generates” will run. It processes text in a extremely sophisticated yet also quite dumb fashion
- It would be convenient if Cheppy would be better at sharing its sources for information – so we can go to that source to verify and explore; unfortunately that is not the case
- An obvious limitation with Cheppy is the actuality of its information – no data from after Summer 2021. In our very fast moving world of cloud services, technology releases, security threats – this means that we cannot rely (solely) on Cheppy to provide current information (especially when sometimes it just guesses – and then profusely apologized when caught in the act)
- Cheppy seems very good at text (duh!): translating into a different language, polishing up, explaining in simpler terms, summarizing, documenting source code. Using Cheppy to produce and enhance the text we have to deliver seems an obvious area of benefit
- Cheppy can help in R&D, PoCs and prototyping: using Cheppy we can quickly explore new technologies and concepts, especially when the code we create does not have to be production quality but merely show (how) something works or our understanding of a concept is only required a fairly high level
- Integration of Cheppy in a browser (and search engine), text editor (Word, VS Code, ..) and in our IDE (VS Code, IntelliJ IDEA, ..) is something we look forward too as it will bring the power even closer to our fingertips
- customer data and customer specific code or documentation should not be sent to Cheppy; it is unclear what may be done to such information and where it may end up. Just like postings on StackOverflow are anonymized, we will treat Cheppy in the same way. (however, code that is not business sensitive and that cannot be traced to a customer through naming or other references can be fed into Cheppy, just like we could share that in other channels); using Cheppy to anonymize sensitive data or code is currently not considered a good idea
We will continue to assemble and share Cheppy use cases that help us in our daily work. We believe that we can now benefit from Cheppy to do things more quickly and and better and sometimes take on challenges that would otherwise have been just out of reach. We will learn how to leverage LLMs like Cheppy and prepare for more advanced successors. We think Cheppy will help in doing the chores that not always today get the attention they deserve (test cases, documentation, translation) and thereby increase the quality of our deliverables.
We expect Cheppy to become a powertool for all our colleagues in the next 3-6 months. In the short term, we will make Cheppy (paid subscription) available to any colleague who asks for it – and we will actively collect experiences (good and bad).
We will explain our customers how we use Cheppy to improve the work we do for them and share our experiences to empower them as well. And we are looking forward to a Cheppy-derivative that we can train on our own data: all our wikis, source code, documents, team chats, blog articles, Yammer, and business emails and then question like we can question Cheppy sounds like a great step in making our internal knowledge management that much easier.
ChatGPT is not a threat, It is an opportunity. It will help us at AMIS in quality and productivity. It will enforce our skills and experience – and that together with our mindset of embracing “the new” will propel us. We do not fear that new competition will arise – from people unskilled, unexperienced that can wield this powertool and then overtake us. Adopting Google search in the late nineties gave good developers superpowers compared to when memory and paper manuals were their source of information – but did little to make life easier for both poor searchers and unskilled engineers. At this moment, Cheppy is in a similar position.
We can benefit as a company because we are in an excellent position to unleash the potential of Cheppy (ChatGPT) and the successors that we can expect over the next coming months.
*) – Fun fact (from Wikipedia): DALL-E is taken from the robot WALL-E and the Spanish artist Dalí.
