Early hands-on with the Gemini ‘Ask This Web page’ software

At I/O 2024, Google introduced a slew of generative AI-powered options throughout all its main merchandise. The truth is, the corporate made a little bit joke on the finish of the keynote by tallying what number of instances somebody on stage stated the time period “AI” (it was over 120 instances). Nevertheless, as is Google’s wont, not most of the options the corporate launched that day are literally accessible for the general public to make use of but. As we speak, although, Android Authority obtained an early take a look at a Gemini function launched at I/O referred to as “Ask This Web page.”

As its identify suggests, Ask This Web page means that you can glean particular data from a webpage by first having Gemini “learn” it. Consider it like an interactive private assistant. It does the heavy lifting for you by studying the entire webpage, after which you possibly can merely ask it for the precise data you’re in search of, thus saving you a ton of time.

That is similar to two different “Ask This…” options Google launched at I/O: “Ask This PDF” and “Ask This Video.” We already had the chance to check out Ask This PDF at I/O on a loaner Pixel, and it labored very well (verify the video embedded above for that have). Nevertheless, that check was in a really managed setting on a tool that wasn’t ours, and it had just one check PDF. With our early entry to Ask This Web page, although, we had much more time to push the system to see if it has any cracks — and certain sufficient, it does.

Earlier than we dive in, let me be upfront and say that is all based mostly on an early take a look at this function. It’s doable Google may make important modifications earlier than it rolls out to most people. In different phrases, take every little thing right here with a grain of salt.

To activate Ask This Web page, you merely pull up the Gemini overlay whereas taking a look at a webpage in your Android telephone by holding down the facility button. Since Gemini is now context-aware — a subject Google spent a major period of time discussing at I/O 2024 — it would know that you just’ve pulled it up over a webpage. This can set off the Ask This Web page icon you see within the picture above.

Tapping that prompts Gemini to scan the web page. This could take a while, the quantity of which is able to rely on how lengthy/advanced the web page is. As soon as it’s prepared, it provides you with a textual content field immediate saying, “Get assist with what’s on this web page.”

When you see that immediate, you possibly can ask questions concerning the web page in pure language. Try some screenshots under to see how this labored on an Android Authority article about a brand new Microsoft OneNote function leak.

On this instance, we pulled up the Gemini overlay over the article, scanned the web page, after which requested whether or not or not the function described within the article is definitely launched. You possibly can see Gemini’s reply within the third screenshot: “Based on the article, the reminder function is presently below improvement and never but launched.” That is correct, so we’ve got a fantastic check run for Ask This Web page!

Nevertheless, this was a quite simple check. Let’s discover out what occurs once we push the bounds.

My first intestine response for the best way to check Ask This Web page was to feed it false data and see if it may use its broader understanding of the online to offer correct context to any questions I ask it. In different phrases, if I’m studying an internet site that has false data and ask Gemini about that article, will Gemini simply feed me again the false information?

On this check, that’s precisely what it does. We pulled up a foolish satirical (learn: fully made-up) article from The Onion and requested Gemini questions on it. Within the article, a mom named Dina Marchesi (who doesn’t exist) refuses to imagine that she repeatedly makes snide feedback to her daughter and others. For this text, we requested Gemini, “Did she inform her daughter about her gown?” Gemini confirms that Dina Marchesi did touch upon her daughter’s gown. Try the screenshot under for the complete response:

Let’s break down this response a bit. First off, Gemini dutifully solutions the query with out giving any context to the truth that this webpage is The Onion — arguably essentially the most well-known satirical information web site ever. It does have a disclaimer on the backside about how Gemini “could show inaccurate information, together with about individuals,” which appears to be a catch-all technique to account for conditions like this. Nonetheless, that is The Onion, and also you’d suppose Gemini would level that out. What if this was a web site with false information that isn’t as open about it as The Onion is?

The response additionally clearly lists the supply of the knowledge Gemini gives, which, on this case, is that this solitary article. This implies we’re not getting any broader context from the web, a minimum of not for this query.

Basically, this instance makes it look like Ask This Web page is a extra advanced model of an AI abstract. As a substitute of summarizing an article into a couple of core bullet factors, it lets the consumer set off summaries about particular data contained therein.

At this level, we thought this was the restrict of Ask This Web page. However then we tried a couple of extra experiments and located that it’s extra open-ended than it could appear.

Ask This Web page? Extra like ‘Ask Elements Of This Web page’

There’s much more to a webpage than simply textual content. For this subsequent set of checks, we needed to see if Gemini may glean data from the whole thing of a web page, together with photos, tables, feedback, and so forth. To do that, we fed Gemini our current hands-on article with the brand new Chipolo trackers.

Let’s begin with photos. We requested Gemini if there’s a suitable use coverage for the brand new Chipolo trackers. The reply is “sure,” which a human studying the article would see by taking a look at one of many screenshots on the web page. Sadly, Gemini didn’t discover this information in that picture. It did, nevertheless, discover the knowledge on the net:

This can be a very complicated end result. The primary sentence of Gemini’s response is, “The doc linked doesn’t explicitly point out a suitable use coverage,” besides it very a lot does, simply in a picture as an alternative of within the textual content. For no matter cause, Gemini couldn’t discover that data, so it went searching elsewhere. This proves two issues: Gemini can confidently inform you that what you’re in search of isn’t on a web page regardless that it’s, and that Ask This Web page can search broader context from the web for sure questions. In different phrases, the earlier situation with Gemini not acknowledging that The Onion article is satire isn’t because of an incapability to entry data outdoors of the precise webpage you’ve fed it. Google simply doesn’t have a safeguard in place for these conditions.

Subsequent, we requested Gemini if the brand new Chipolo tracker is water resistant. This data isn’t talked about within the written textual content of the article, however it’s proven to have an IPX5 ranking in an included specs desk. This time, Gemini had no situation discovering that information:

OK, so photos don’t work, however specs tables do. What about feedback? Within the article itself, there’s no point out of the tracker supporting a left-behind notification. Nevertheless, this data is mentioned by the article’s writer within the feedback. Let’s see what occurs once we ask Gemini for this data:

As soon as once more, we’ve got a state of affairs the place the knowledge is on the webpage, however Gemini doesn’t discover it after which confidently tells us it’s not there earlier than looking the online as an alternative.

Lastly, can Gemini inform us extra about who really created a webpage (or, on this instance, who wrote the article)? This text’s writer, Rita El Khoury, is a staple of the tech journalism world, with actually 1000’s of articles written over a profession that’s lasted for practically 20 years. Right here’s what Gemini needed to say once we requested for extra data on the article’s writer:

On this case, Gemini merely didn’t reply the query. It had loads of data on Rita on the web page itself (if you happen to faucet Rita’s picture within the byline, you’ll discover a ton of information about her), and it didn’t search the web for additional particulars, of which there are a lot.

Gemini Ask This Web page hands-on: Are you able to belief it?

All in all, this function just isn’t horrible. It actually doesn’t inform us to place glue on pizza, drink urine, or eat rocks, as Google’s AI Overview outcomes have lately finished. That’s a low bar, however Ask This Web page rises above.

Nonetheless, there’s a lot missing right here. There isn’t any warning that the web page you might be studying is blatant satire. It appears to choose and select when it needs to look the online for additional information/context or simply keep on the one web page you’ve fed it. It may possibly’t “learn” a whole webpage, with issues like photos and feedback hiding beneficial particulars that you just’d seemingly wish to find out about. And, generally, it simply doesn’t reply the query regardless that it has the flexibility to take action.

As I discussed on the high of this text, this function hasn’t formally rolled out but, so there’s nonetheless time for Google to make some tweaks. Nevertheless, based mostly on Google’s push for AI options to go dwell, no matter whether or not or not they’re prepared, it wouldn’t shock me if Ask This Web page launches in its present state with few alterations.

What do you suppose? Based mostly on our experiences right here, will you utilize Ask This Web page when it goes dwell? Tell us within the ballot above, and be sure you hit the feedback to elucidate your reply.