Inspired by this post by Simon Willison, I’ve been playing a little bit with Apple Photos database and Simon’s dogsheep-photos utility.

Somehow this project spiralled into redoing the way photos are stored on this blog, which is probably a whole different post, and ended up being where I spent most (probably too much) of my time.

The two aspects that interested me the most were the auto-generated ML tags and the overall aesthetic score. I think one of the things that separates hobbyist photographers from professional photographers is a critical eye on pictures already taken. When I’m sorting photographs, I usually do a gut check “Do I like this” followed by “Is this anything?”. I’m not sure there’s any formal training or artistic merit to those two feelings.

I do find the camera synthesis Apple does on the phone really impressive, their photo widget that randomly pops up pictures from your camera roll is fantastic.
They definitely optimize for pictures of people having fun, though, which is perfect for a phone widget and not really “Here’s are the good pictures you’ve taken, artistically.”

According to Apple, this is the highest photo in my photos database for overall aesthetic, taken at the Georgia Aquarium:

I’m not 100% sure I agree, but it’s definitely a nice photo of a jellyfish.

The ML tagging itself is pretty banal (adult, child, mammal, travel, luggage), which makes it a bit strange to look at individually. However, there is a lot of different training tags - 948 in my database. And some very specific, like “Arugula” or “Road Safety Equipment”. I’m not sure you can do much with these tags in isolation, but a web UI to explore by content might make sense. I’d also be curious to compare Apple’s ML vs Amazon’s Rekognition vs Google ML.

Going over this entire set (giong back to college 20 years ago!), you can see where I spend most of my time. I have a lot of pictures of mountains, and since I cook/bake a lot a lot of pictures of food. I’m not sure the ML library digs pictures of food, since it rates the mountains much higher.

Of course the ML misses are almost all hilarious. This Chihuly statue is labelled a plant:

Mostly reviewing these turned into a cool trip down memory lane, and in most cases I said, “You’re right, I like that picture”

For instance, here’s a pretty solid picture labelled “bookshelf”, because Powell’s is amazing.

I’ve made a lightly edited list of the Top 100 items by overall aesthetic score at Top 100. Overall, it’s kind of a cool list of photos that I’m proud of.