Something a little different on a sea day. Exploring photos using AI
These are the photos I was looking for, and this is the story of how I used the new AI feature in Google Photos to find them.
My wife and I are about to set out on a month-long adventure that will take us to Alaska and then Japan. The strange combination is due to what is called a boat “repositioning” by the cruise industry. This is the fifth trip we have made to Alaska starting with driving the ALCAN highway when some of it was still gravel, several ferry trips through the inside passage, and now a cruise ship.
As I prepared for this trip, I tried to locate images from past visits. I have always taken photos and written about our trips. Still, this content was collected over nearly fifty years and was created with multiple cameras and phones and scattered across online storage, numerous computers, and various types of storage media. When I retired, I undertook a project to collect as many images as possible in one location. I chose Google Photos, mainly because the images would automatically upload as I found them.
My situation is not unusual for someone of my age. If you are an older individual, you already probably know that your kids don’t want your stuff. They also are not that interested in your pictures. Even those in which they appear. This does not mean that you won’t appreciate the opportunity to review past experiences. I find that selecting a few and generating the stories that go with them is more likely to generate interest in others.
Back to the bears.
During one of our Alaska trips, we visited our daughter and her future husband, who were spending the summer exploring and working on a whale watching boat. The daughter was in a phase where she took a lot of photos and even blogged about her experiences. We were walking on the shore and she was using my camera when we saw several bears, one of which was out in the water. She captured a series of images as the bear moved through the water and at one point, climbed and posed on a rock before moving on. The bear on the rock picture garnered significant attention and appeared on the front page of the Ketchikan paper. I had the famous photo, but not the sequence, and that is what I wanted to see if I could find.
Here is the challenge. My collection includes thousands and thousands of photos. Over the years, I have paid for and maintained a collection in Flickr that is made up of more than 7000. The issue with Flickr was that I had used it for many years to showcase my high-quality shots. I was not using the service to collect the large number of images I deemed of lesser interest to others. The other images accumulated in iPhoto on several machines and perhaps as slides or digital photos stored on CDs or whatever happened to be the external media of the day. As I rediscovered these images, they ended up in Google Photos in no particular order. Note, few of these images had geolocation data, and many not even the EXIF data that would indicate the date the images were taken. Cameras, an SLR in this case, do not have the means to record the geolocation data we expect from cellphones.
Here is the feature Google has added to photos that seems added just for my situation. You can now ask Gemini (AI), available within Photos, to perform a wide range of tasks based on your photo collection. First, the Ask feature does not presently work on a Macintosh, but does work with the Google Photos app on the iPhone or iPad. Ask appears at the bottom of the screen instead of the search icon, which may be confusing if you are used to the traditional search method. Entering the prompt “Find images of bears in my photo collection” creates a long list of hits - bears in zoos, a photo of a black bear carved from a tree stump, teddy bears being held by our kids, and all of the bear images taken in Alaska. I was able to locate one of the missing bear images as the bear approached the rock I could not find in any other way.
I tried one more search that also produced some very surprising results. I knew that we had fished for salmon on one of our Alaska trips and searched for salmon. I found photos I had taken of salmon in restaurants, pictures of smoked salmon within a display case I took while visiting the Upper Peninsula of Michigan, and pictures of salmon taken while fishing in Alaska. The interesting thing about the fishing pictures was that these images appeared not as photos, but within videos. No tags, no meaningful file labels - AI found the salmon within the videos.
How can this work? The only approach that seems reasonable is that Google “processes” stored content assigning what I might called tags to things and actions associated with blocks in files and frames within videos. A prompt then uses these existing tags to generate a response. So, I am speculating that Google preprocesses the content you give it for storage much like say the content you upload for use in NotebookLM.
Summary
Google has added AI to specific implementations of Google Photos. The AI allows intelligent search that is not dependent on stored text, dates, or geolocation data. The AI can use these sources, but can also identify imagery directly. This capability is not presently available on all devices, but works via a device that makes use of apps (iOS or Android).
No comments:
Post a Comment