Taxonomy, Google Gemini, and organism identification from images

In Nature Volume 652 from 16 April 2026 on page 543 Chris Bivins argues that the world needs more taxonomists. One particular line in the article caught my attention, "But when I find a fungus that lacks a name, I can still hold it, examine it under a microscope, compare it to known species and begin the work of description. An LLM cannot. It has no access to the physical world, only to text." 

That LLM's only have access to text is simply not true. LLM's are now quite proficient at image analysis. Give the LLM images of known fungal species and the LLM can begin the work of description, potentially highlighting similarities and differences to known species.

I gave Google Gemini app this image:


And the prompt "Describe this photo along with the likely identification of the organisms in the photo."

The response started at the blue diamond:

This image captures a close-up look at a brush-footed butterfly resting amidst lush green foliage.

Visual Description

The Butterfly: Positioned in the upper center of the frame, the butterfly is viewed from a slightly angled ventral (underside) perspective as its wings are partially closed. The visible wing surfaces are predominantly deep brown with a prominent, jagged white band running diagonally across the wing, complemented by smaller white spots and intricate patterns along the edges.

The Foliage: The background and foreground feature large, broad green leaves with visible veining. Directly beneath the butterfly is a cluster of small, pale green, un-blossomed flower buds, which appear to belong to the host plant or shrub it is resting on.

Likely Organism Identification

The Butterfly: This insect is highly likely a female Great Eggfly (also known as the Common Eggfly or Blue Moon Butterfly, scientifically classified as Hypolimnas bolina). While the males of this species feature striking, velvety-black wings with glowing ultraviolet-blue and white spots on their upper side (dorsal view), the females are incredibly variable and typically present a more muted, cryptic brown underside with distinct white banding to blend in with foliage when resting.

The Plant: The broad, glossy green leaves and the distinct dense, branching clusters of tiny green flowers are highly characteristic of a Premna species (such as Premna serratifolia or Premna odorata). These coastal shrubs are common across the tropical Indo-Pacific and are known to be a favorite nectar source and resting site for butterflies like the Great Eggfly.

~~~

Although the photo is far from ideal, Google Gemini correctly identified the butterfly to species. Gemini's lead response for the plant is also correct. That Gemini picked off the host plant from a suboptimal image of the plant is nothing short of amazing. Not to mention an estimate of the gender of the butterfly. AI is certainly not limited to text. 

A few years ago I was complaining about the taxonomic churn in Polypodiopsida. Ferns were moving to new genera and species epithets on an annual basis. A field biologist reminded me that anything above the species level is a human construct that is subject to change as understandings and frameworks change. Perhaps AI cannot replace human taxonomists, but then AI may develop its own internal taxonomic framework. The AI undoubtedly does not see and understand the image in the exact same way as a human expert. Yet they both converge on the same set of possible organisms. At some point AI will be able to determine when the number of differences from known organisms are sufficient that the organism warrants a new scientific name. 

In some philosophical sense, neither humans nor AI have access to the physical world. For both access is mediated by inputs. Currently humans have more input modalities - vision, hearing, taste, smell, and touch. AI is generally limited to visual inputs. Yet this will also change. AI integration into robotics will bring more inputs. AI sensors might eventually include real time environmental DNA sensors (see DNA in the air is unlocking ecosystems' secrets in the same issue of Nature cited earlier).  At that point AI will be a superior taxonomist to any human being. 

Comments

Post a Comment

Popular posts from this blog

Setting up a boxplot chart in Google Sheets with multiple boxplots on a single chart

Image
The  Google Statistics  add-on for Google Sheets can display multiple boxplots in a single chart. The key is the layout of the data. One column should be the variable by which the data is to be grouped, the other column should be the data to be box plotted. Set up the Statistics add-on with the data to be plotted as the variable, and the grouping column as the "by" variable. In this image I had deselected all but the boxplot option, the result was the appearance of the Moment, Standard errors, and Confidence intervals options. The default is apparently a 95% confidence interval for the mean. The result is multiple boxplots on a single chart with a common scale. The new tab that is created also quotes 95% confidence intervals for the mean. Note that as of 2018 the Google Statistics add-on cannot be found by search in the add-ons. In addition, as of May 2018 the add-on no longer verifies, possibly due to the add-on not having been updated since August 2017. One may ha...
Read more

Traditional food dishes of Micronesia

Image
Clara, Jennette, and Joanie presented ground hard taro cooked in an umw, effectively a hot rock oven. Pohnpeian: rotama. Pingelapese: sero. Mwoakillese: rodma. Mortlockese: Amahd. Kosrae: rodoma. After cooking in an umw the rotama is pounded with the petiole of a palm frond. Rotama is made from hard or swamp taro, known as mwahng on Pohnpei, mweian on Pingalap, pula in Mortlockese, and pahsruhk on Kosrae. In traditional times on the outer islands the women had primary responsibility for tending the taro while the men handled fishing and climbing tasks. Joesen and Noeleen presented fermented breadfruit from the outer islands of Yap such as Woleai and Eauripik. The raw ingredients were actually brought in from Yap, Noeleen prepared the maare.  The leaf wrap is ti leaf - Cordyline fruticosa. Pauleen, Barnson, Trisha, Verginia, Con-ray, and Maylanda from Kitti, Pohnpei brought in mahi umw, koahpnoair koakihr, and uht sukusuk. Above is breadfruit umw, mahi umw, o...
Read more

Plotting polar coordinates in Desmos and a vector addition demonstrator

Image
In an earlier post I noted that Desmos did not directly plot polar coordinates. Not only was I incorrect, but Desmos responded to my blog to correct me! Although I had at some point seen that one could define a function f(x)=3x+5 and then have Desmos calculate f(6), I had not absorbed how this might be used to plot polar coordinates. The above works beautifully . Realizing that I could effectively program Desmos, I applied this thinking to demonstrating how to add two vectors when given the magnitudes m and the direction angles theta. The graph calculates the i and j components for the two vectors and then adds the vectors, graphically displaying the result while also providing information on the magnitude and direction of the vector sum. m1 and theta1 are the magnitude and direction angle for one of the two vectors, m2 and theta2 are the magnitude and direction for the other vector. All four are dynamically interactive and can be changed. The diagram purports to illustra...
Read more