The Advanced Voice Mode of Chat GPT can be used with real-time vision

Over the course of 12 days, OpenAI is hosting its own version of an advent calendar, where the surprises are updates about its products. The latest reveal is the integration of vision capabilities in ChatGPT’s advanced voice mode.

Currently, this feature is only available in the ChatGPT app and is being rolled out to users with Plus, Team, or Pro plans. On the other hand, those subscribed to the Enterprise or Edu plans will need to wait until January to access the advanced voice mode with vision.

Regarding the integration of this feature in regions like the European Union, Switzerland, Iceland, Norway, or Liechtenstein, OpenAI has stated they will proceed “as soon as possible”. This delay is likely due to the need to ensure compliance with European regulations before launching such a product in the market.

Real-Time Vision for Enhanced Interaction

The new vision capabilities allow you to share live-recorded videos or live screen recordings with the ChatGPT app’s chatbot. The AI can understand the environment and elements you focus on, and then respond to questions or perform actions based on this visual data.

OpenAI representatives conducted a live demonstration during the announcement broadcast. For example, ChatGPT was tasked with guiding Rowan, an OpenAI researcher, by providing step-by-step instructions to make coffee in real-time.

Just in time for the holidays, video and screensharing are now starting to roll out in Advanced Voice in the ChatGPT mobile app. pic.twitter.com/HFHX2E33S8

— OpenAI (@OpenAI) December 12, 2024

How to Use ChatGPT’s Advanced Voice Mode with Vision

To use vision in the advanced voice mode, follow these steps:

1. Open the ChatGPT app and start a conversation.
2. Click on the “voice mode” button located at the bottom right of the ChatGPT message window.
3. Once in voice mode, you can:
   1. Select the video camera icon to start talking to ChatGPT as if you were on a live video call.
   2. Choose the three-dot dropdown menu and click on “share screen” to share your screen live. You can also select “upload photo” or “take photo”.

ChatGPT Celebrates the Holidays with “Santa Claus Voice Mode”

The vision integration in advanced voice mode was not the only surprise, as OpenAI also unveiled a festive feature: the “Santa Claus mode.” This allows you to choose the voice of Santa Claus when using voice mode, so you can interact with this jolly character.

Say ho ho ho to Santa in Voice Mode

Santa is rolling out today to everyone across all ChatGPT platforms and is available until the end of the month…then he will retreat back to the North Pole. pic.twitter.com/NVS9bRok4r

— OpenAI (@OpenAI) December 12, 2024

To use “Santa Claus voice mode,” simply click on the snowflake icon next to the message bar in the bottom right corner. Other options include accessing ChatGPT settings or using the voice selector located in the upper right corner within the voice mode itself.

Photo: OpenAI