Which is the better and smarter AI chatbot?

After playing catchup for months, Google has finally launched its first commercial generative AI product in the form of Gemini. Available in both free and paid versions, Google Gemini is a chatbot and AI image generator that you can also use as a digital assistant on Android instead of the Google Assistant. But can it outperform other chatbots and convince us to make the switch? Let’s find out in this Gemini vs ChatGPT head-to-head comparison.

The free version of Google’s chatbot relies on the Gemini Pro language model. It’s the middle-sized version of the company’s model. The free version of ChatGPT, on the other hand, uses OpenAI’s GPT-3.5 language model. In either chatbot’s case, you can pay to unlock better models and improve output quality.

Since most users won’t pay to upgrade, let’s start by comparing Gemini Pro vs ChatGPT 3.5. It’s worth noting that Google’s chatbot can access the internet for up-to-date information. On the other hand, ChatGPT has a hard knowledge cut-off and doesn’t know about recent events. It cannot fact-check its information either, so the free version of ChatGPT isn’t a good choice if you need to ask about anything current.

With that disclaimer out of the way, let’s see whether Gemini and ChatGPT can perform simple coding tasks. First off, I asked both chatbots to generate code for a website that shows the current trading price of the S&P 500 index.

ChatGPT spat out a code block without hesitation, along with some instructions to fetch a free API key from an online stock data provider. On the other hand, Gemini refused to complete the task and simply said “I cannot directly generate code for you due to limitations in my capabilities.” I then deleted the chat and sent the same prompt. This time, Gemini generated two code blocks.

I saved both chatbot’s results as instructed and opened them in a web browser. ChatGPT’s output worked in the first go, but Gemini’s code failed to produce any results. I promptly fed the error back into Google’s chatbot but it gave me a list of generic reasons. As a final resort, I asked it to use the same API as ChatGPT. It failed yet again, so I gave up at this point.

Next, I fed both chatbots the known working code (the one from ChatGPT) and asked them to modify it. More specifically, I asked for a list of 10 random stocks instead of just one along with an improved design. This task should eliminate the previous test’s third-party variables and allow us to gauge each chatbot’s inherent ability to generate code. Here are the results:

Overall, both chatbots created a decent-looking website with a list of ten stock prices. However, I had to jump in and correct ChatGPT’s code manually in one or two places as it used an invalid API link. This is likely the result of ChatGPT’s inability to access the internet. Gemini, on the other hand, admitted that it read through someone else’s project on GitHub before generating its own code. This is a good thing to be clear, as long as no plagiarism was involved.

Last year, ChatGPT had a massive upper hand when it came to coding but I’m happy to report that Google has stepped up its game. Gemini’s ability to search the internet is also genuinely useful and makes it the better choice in many research-oriented use-cases. However, neither chatbot will do complex jobs entirely for you.

The Gemini platform has one trick up its sleeve compared to ChatGPT: it can read your data from various Google apps. In the above screenshots for example, I asked Gemini to access my email and calculate how much I’ve spent on food delivery this year. Once it listed all of the individual amounts, I asked for a table in a follow-up prompt and exported the data to a Google Sheets spreadsheet.

While some of this is possible using ChatGPT, it’s nowhere near as seamless. Not to mention, you don’t get access to any ChatGPT plugins as a free user. The Gemini extensions feature, meanwhile, is available for free.

Both Google and OpenAI offer paid versions of their respective chatbots, which unlocks higher-quality responses and other features. The main draw for ChatGPT Plus is the ability to search the internet via Bing alongside the newer GPT-4 Turbo language model. Google’s subscription, on the other hand, gives you the Gemini Ultra model as well as AI-driven features in Workspace apps like Gmail.

Even though both subscriptions will set you back $20 per month, Gemini comes out ahead from a value perspective. The Google One AI Premium subscription also grants you 2TB of cloud storage along with other perks like a free VPN. It’s also available as a free trial for two months, whereas ChatGPT’s subscription will bill you on day one.

Of course, value doesn’t matter as much as the quality of the results. So let’s evaluate Gemini Ultra vs ChatGPT 4 to see which one comes out on top.

Starting with a factual question, I asked both chatbots which countries get the Exynos variant of Samsung’s Galaxy S24 smartphone. Unsurprisingly then, both chatbots responded perfectly and even specified which countries will receive Exynos variants. We can call this round a tie.

Moving on, let’s test the vision capabilities of both chatbots. I uploaded an image of four framed art pieces and asked both chatbots whether they could interpret the meaning behind them. Here’s how they responded:

Gemini clearly stumbled a bit here, it responded as if the images were in a square grid rather than in a horizontal sequence. Still, it arrived at the same (correct) conclusion as ChatGPT and that’s what matters in the end. But here’s a twist: I used the free version of Gemini for the above result! ChatGPT doesn’t offer such image analysis unless you have an active subscription.

Google hasn’t activated Gemini Ultra’s multimodal features as of the time of writing this article, even for paying customers, so a direct comparison with GPT-4 couldn’t be made.

So far, we know that both Gemini and ChatGPT offer accurate responses and exhibit similar levels of intelligence. However, Google’s chatbot does offer many more features if you’re not ready to spend $20 per month on an AI platform. The ability to search the internet is a big game changer in its own right.

Gemini also offers a more seamless experience on Android since you can set it as your default assistant. It replaces the Google Assistant when you do so and can even perform basic tasks like setting alarms. It’s under active development for now, but expect it to become a whole lot more competent in the coming months and years.

Having said that, the pendulum swings back in ChatGPT’s favor if you need a completely hands-free chatbot experience. The ChatGPT app offers a voice chat mode that’s second to none. It delivers an immersive conversational experience that you won’t find anywhere else.

You may not like hearing it but the answer to which chatbot you should use will boil down to your individual needs. Creatives will find value in ChatGPT and its generous character limit, for example, while those within the Google ecosystem will probably be better off using Gemini. Which AI chatbot would you rather use?