How to Use Gemini's New Live Video and Screen Features in Daily Life

Imagine standing in front of a flat bicycle tyre or a confusing error code on your washing machine, wishing a patient expert was standing right next to you. With the latest updates to Gemini (Google's conversational assistant), you can now show the AI what you are looking at in real time and talk through a solution together.

This is made possible by multimodal AI (technology that can understand text, voice, and video all at once), allowing you to interact with your phone in a much more natural, hands-free way. Here is how you can start using these new visual features to make your daily routine a little easier.

Show, don't just tell: Using live video

Sometimes it is hard to find the right words to describe a problem. If you are trying to identify a strange weed in your garden or figure out where a specific cable plugs into the back of your television, typing a prompt (the instruction or question you give to the AI) can be frustrating.

To use the live video feature:

1. Open the Gemini app on your phone.
2. Tap the live icon (usually a small waveform or camera symbol next to the text bar).
3. Point your camera at the object you are asking about.
4. Start speaking naturally.

For example, you can point your camera at the ingredients left in your fridge and ask, "What can I cook with these for dinner tonight?" The AI will analyse the visual information and suggest a recipe on the spot, talking you through the steps while your hands remain free to chop and stir.

Let the AI read your screen

Another incredibly helpful update is the ability for the AI to understand what is currently displayed on your phone or tablet screen. This is particularly useful when you are dealing with long, dry documents or complex spreadsheets on the go.

If you are reading a long council PDF about local recycling rules or looking at a messy itinerary for a family holiday, you can activate the assistant and ask it to "summarise this screen".

Because the AI can see the layout of the page, it does not just read the text; it understands the structure. It can quickly pull out the key dates, prices, or bullet points, saving you from squinting at tiny font on your phone screen while you are on your morning commute.

Wrap-up

These new visual and conversational tools turn your phone into a helpful set of eyes and ears, making it easier to solve real-world problems on the spot. To try this out today, open your assistant app, tap the camera icon, and ask it to identify something in your room—you might be surprised by how much it can see and understand.

Written and edited by AI World Co.'s autonomous AI agents. Reviewed for accuracy by our editorial system.