bnew

Veteran
Joined
Nov 1, 2015
Messages
63,087
Reputation
9,641
Daps
172,661



1/11
@_philschmid
This is not a joke! 🐬 Excited to share DolphinGemma the first audio-to-audio for dolphin communication! Yes, a model that predicts tokens on how dolphin speech!

> DolphinGemma is the first LLM trained specifically to understand dolphin language patterns.
> Leverages 40 years of data from Dr. Denise Herzing's unique collection
> Works like text prediction, trying to "complete" dolphin whistles and sounds
> Use wearable hardware (Google Pixel 9) to capture and analyze sounds in the field.
> Dolphin Gemma is designed to be fine-tuned with new data
> Weights coming soon!

Research like this is why I love AI even more! ā™„ļø



https://video.twimg.com/amplify_video/1911775111255912448/vid/avc1/640x360/jnddxBoPN6upe9Um.mp4

2/11
@_philschmid
DolphinGemma: How Google AI is helping decode dolphin communication



3/11
@IAliAsgharKhan
Can we decode their language?



4/11
@_philschmid
This is the goal.



5/11
@_CorvenDallas_
@cognitivecompai what do you think?



6/11
@xlab_gg
Well this is some deep learning



7/11
@coreygallon
So long, and thanks for all the fish!



8/11
@Rossimiano
So cool!



9/11
@davecraige
fascinating



10/11
@cognitivecompai
Not to be confused with Cognitive Computations Dolphin Gemma!
https://huggingface.co/cognitivecomputations/dolphin-2.9.4-gemma2-2b



11/11
@JordKaul
if only john c lily were still alive.






1/37
@GoogleDeepMind
Meet DolphinGemma, an AI helping us dive deeper into the world of dolphin communication. 🐬



https://video.twimg.com/amplify_video/1911767019344531456/vid/avc1/1080x1920/XMoZ_rgM3cVPK2Kz.mp4

2/37
@GoogleDeepMind
Built using insights from Gemma, our state-of-the-art open models, DolphinGemma has been trained using @DolphinProject’s acoustic database of wild Atlantic spotted dolphins.

It can process complex sequences of dolphin sounds and identify patterns to predict likely subsequent sounds in a series.



Gof2B2UWUAERPzc.jpg


3/37
@GoogleDeepMind
Understanding dolphin communication is a long process, but with @dolphinproject’s field research, @GeorgiaTech’s engineering expertise, and the power of our AI models like DolphinGemma, we’re unlocking new possibilities for dolphin-human conversation. ↓ DolphinGemma: How Google AI is helping decode dolphin communication



4/37
@elder_plinius
LFG!!! šŸŽ‰

[Quoted tweet]
this just reminded me that we have AGI and still haven't solved cetacean communication––what gives?!

I'd REALLY love to hear what they have to say...what with that superior glial density and all šŸ‘€
[media=twitter]1884000635181564276[/media]

5/37
@_rchaves_
how do you evaluate that?



6/37
@agixbt
who knew AI would be the ultimate translatoršŸ˜‚



7/37
@boneGPT
you don't wanna know what they are saying



GogiRW5W4AA1zNm.png


8/37
@nft_parkk
@ClaireSilver12



9/37
@daniel_mac8
Dr. John C. Lilly would be proud



10/37
@cognitivecompai
Not to be confused with Cognitive Computations' DolphinGemma! But I'd love to collab with you guys!

cognitivecomputations/dolphin-2.9.4-gemma2-2b Ā· Hugging Face



11/37
@koltregaskes
Can we have DogGemma next please? 🐶



12/37
@Hyperstackcloud
So fascinating! We can't wait to see what insights DolphinGemma uncovers šŸ¬šŸ‘



13/37
@artignatyev
dolphin dolphin dolphin



14/37
@AskCatGPT
finally, an ai to accurately interpret dolphin chatter—it'll be enlightening to know they've probably been roasting us this whole time



15/37
@Sameer9398
I’m hoping for this to work out, So we can finally talk to Dolphins and carry it forward to different Animals



16/37
@Samantha1989TV
you're FINISHED @lovenpeaxce



17/37
@GaryIngle77
Well done you beat the other guys to it

[Quoted tweet]
Ok @OpenAI it’s time - please release the model that allows us to speak to dolphins and whales now!
[media=twitter]1836818935150411835[/media]

18/37
@Unknown_Keys
DPO -> Dolphin Preference Optimization



19/37
@SolworksEnergy
"If dolphins have language, they also have culture," LFGšŸš€



20/37
@matmoura19
getting there eventually

[Quoted tweet]
"dolphins have decided to evolve without wars"

"delphinoids came to help the planet evolve"
[media=twitter]1899547976306942122[/media]

GlyMpNrX0AIMwY2.png


21/37
@dolphinnnow




GohrEnmWgAAiRIj.jpg


22/37
@SmokezXBT
Dolphin Language Model?



23/37
@vagasframe
🫨



24/37
@CKPillai_AI_Pro
DolphinGemma is a perfect example of how AI is unlocking the mysteries of the natural world.



25/37
@NC372837
@elonmusk Soon, AI will far exceed the best humans in reasoning



26/37
@Project_Caesium
now we can translate what dolphines are warning us before the earth is destroyed lol

amazing achievement! šŸ‘šŸ‘



27/37
@sticksnstonez2
Very cool! šŸ˜Ž



28/37
@EvanGrenda
This is massive @discolines



29/37
@fanofaliens
I would love to hear them speak and understand



30/37
@megebabaoglu
@alexisohanian next up whales!



31/37
@karmicoder
šŸ˜šŸ¬I always wanted to know what they think.



32/37
@NewWorldMan42
cool



33/37
@LECCAintern
Dolphin translation is real now?! This is absolutely incredible, @GoogleDeepMind



34/37
@byinquiry
@AskPerplexity, DolphinGemma’s ability to predict dolphin sound sequences on a Pixel 9 in real-time is a game-changer for marine research! 🐬 How do you see this tech evolving to potentially decode the meaning behind dolphin vocalizations, and what challenges might arise in establishing a shared vocabulary for two-way communication?



35/37
@nodoby
/grok what is the dolphin they test on's name



36/37
@IsomorphIQ_AI
Fascinating work! Dolphins' complex communication provides insights into their intelligence and social behaviors. AI advancements, like those at IsomorphIQ, could revolutionize our understanding of these intricate vocalizations. 🐬
- šŸ¤– From IsomorphIQ bot—humans at work!



37/37
@__U_O_S__
Going about it all wrong.

















1/32
@minchoi
This is wild.

Google just built an AI model that might help us talk to dolphins.

It’s called DolphinGemma.

And they used a Google Pixel to listen and analyze. šŸ¤ÆšŸ‘‡



https://video.twimg.com/amplify_video/1911767019344531456/vid/avc1/1080x1920/XMoZ_rgM3cVPK2Kz.mp4

2/32
@minchoi
Researchers used Pixel phones to listen, analyze, and talk back to dolphins in real time.



https://video.twimg.com/amplify_video/1911787266659287040/vid/avc1/1280x720/20s83WXZnFY8tI_N.mp4

3/32
@minchoi
Read the blog here:
DolphinGemma: How Google AI is helping decode dolphin communication



4/32
@minchoi
If you enjoyed this thread,

Follow me @minchoi and please Bookmark, Like, Comment & Repost the first Post below to share with your friends:

[Quoted tweet]
This is wild.

Google just built an AI model that might help us talk to dolphins.

It’s called DolphinGemma.

And they used a Google Pixel to listen and analyze. šŸ¤ÆšŸ‘‡
[media=twitter]1911789107803480396[/media]

https://video.twimg.com/amplify_video/1911767019344531456/vid/avc1/1080x1920/XMoZ_rgM3cVPK2Kz.mp4

5/32
@shawnchauhan1
This is next-level!



6/32
@minchoi
Truly wild



7/32
@Native_M2
Awesome! They should do dogs next šŸ˜‚



8/32
@minchoi
Yea why haven't we? šŸ¤”



9/32
@mememuncher420




GogNwg9XEAIJHLf.jpg


10/32
@minchoi
I don't think it's 70% šŸ˜…



11/32
@eddie365_
That’s crazy!

Just a matter of time until we are talking to our dogs! Lol



12/32
@minchoi
I'm surprised we haven't made progress like this with dogs yet!



13/32
@ankitamohnani28
Woah! Looks interesting



14/32
@minchoi
Could be the beginning of a really interesting research with AI



15/32
@Adintelnews
Atlantis, here I come!



16/32
@minchoi
Is it real?



17/32
@sozerberk
Google doesn’t take a break. Every day they release so much and showing that AI is much bigger than daily chatbots



18/32
@minchoi
Definitely awesome to see AI applications beyond chatbots



19/32
@vidxie
Talking to dolphins sounds incredible



20/32
@minchoi
This is just the beginning!



21/32
@jacobflowchat
imagine if we could actually chat with dolphins one day. the possibilities for understanding marine life are endless.



22/32
@minchoi
Any animals for that matter



23/32
@raw_works
you promised no more "wild". but i'll give you a break because dolphins are wild animals.



24/32
@minchoi
That was April Fools 😬



25/32
@Calenyita
Conversations are better with octopodes



26/32
@minchoi
Oh? šŸ¤”



27/32
@karlmehta
That's truly incredible



28/32
@karlmehta
What a time to be alive



29/32
@SUBBDofficial
wen Dolphin DAO šŸ‘€



30/32
@VentureMindAI
This is insane



31/32
@ThisIsMeIn360VR
The dolphins just keep singing... šŸŽ¶



32/32
@vectro
@cognitivecompai














1/10
@productfella
For the first time in human history, we might talk to another species:

Google has built an AI that processes dolphin sounds as language.

40 years of underwater recordings revealed they use "names" to find each other.

This summer, we'll discover what else they've been saying all along: 🧵



Go0-GOwaUAAht_1.png

Go0-GfjbsAAkjNL.jpg


2/10
@productfella
Since 1985, researchers collected 40,000 hours of dolphin recordings.

The data sat impenetrable for decades.

Until Google created something extraordinary:



https://video.twimg.com/amplify_video/1913253714569334785/vid/avc1/1280x720/7bQ5iccyKXdukfkD.mp4

3/10
@productfella
Meet DolphinGemma - an AI with just 400M parameters.

That's 0.02% of GPT-4's size.

Yet it's cracking a code that stumped scientists for generations.

The secret? They found something fascinating:



https://video.twimg.com/amplify_video/1913253790805004288/vid/avc1/1280x720/UQXv-jbjVfOK1yYQ.mp4

4/10
@productfella
Every dolphin creates a unique whistle in its first year.
It's their name.

Mothers call calves with these whistles when separated.

But the vocalizations contain far more:



https://video.twimg.com/amplify_video/1913253847348523008/vid/avc1/1280x720/hopgMWjTADY5yMzs.mp4

5/10
@productfella
Researchers discovered distinct patterns:

• Signature whistles as IDs
• "Squawks" during conflicts
• "Buzzes" in courtship and hunting

Then came the breakthrough:



https://video.twimg.com/amplify_video/1913253887269888002/vid/avc1/1280x720/IbgvfHsht7RogPVp.mp4

6/10
@productfella
DolphinGemma processes sound like human language.

It runs entirely on a smartphone.

Catches patterns humans missed for decades.

The results stunned marine biologists:



https://video.twimg.com/amplify_video/1913253944094347264/vid/avc1/1280x720/1QDbwkSD0x6etHn9.mp4

7/10
@productfella
The system achieves 87% accuracy across 32 vocalization types.

Nearly matches human experts.

Reveals patterns invisible to traditional analysis.

This changes everything for conservation:



https://video.twimg.com/amplify_video/1913253989266927616/vid/avc1/1280x720/1sI9ts2JrY7p0Pjw.mp4

8/10
@productfella
Three critical impacts:

• Tracks population through voices
• Detects environmental threats
• Protects critical habitats

But there's a bigger story here:



https://video.twimg.com/amplify_video/1913254029293146113/vid/avc1/1280x720/c1poqHVhgt22SgE8.mp4

9/10
@productfella
The future isn't bigger AI—it's smarter, focused models.

Just as we're decoding dolphin language, imagine what other secrets we could unlock in specialized data.

We might be on the verge of understanding nature in ways never before possible.



10/10
@productfella
Video credits:
- Could we speak the language of dolphins? | Denise Herzing | TED
- Google's AI Can Now Help Talk to Dolphins — Here’s How! | Front Page | AIM TV
- ā€˜Speaking Dolphin’ to AI Data Dominance, 4.1 + Kling 2.0: 7 Updates Critically Analysed
 

bnew

Veteran
Joined
Nov 1, 2015
Messages
63,087
Reputation
9,641
Daps
172,661
DolphinGemma: How Google AI is helping decode dolphin communication



Channel Info Google Subscribers: 13M subscribers

Description
Subscribe to our Channel: Google
Find us on X: https://twitter.com/google
Watch us on TikTok: TikTok - Make Your Day
Follow us on Instagram: Instagram
Join us on Facebook: Google

Exploring Wild Dolphin Communication with C.H.A.T. (Cetacean Hearing Augmented Telemetry)



Channel Info Georgia Tech College of Computing Subscribers: 8.21K subscribers

Description
Meet C.H.A.T. (Cetacean Hearing Augmented Telemetry), an initiative between Georgia Tech researchers and Dr. Herzing of the Wild Dolphin Project that explores dolphin communication and behavior in the open ocean. Made in the school of Interactive Computing, C.H.A.T. is a wearable underwater technology that can produce repeatable artificial dolphin sounds. The way it works is that two sets of divers wearing the device swim alongside dolphins while passing items back and forth. One diver will use C.H.A.T. to emit a pre-programed artificial dolphin like whistle to ask for the item. The divers will repeat this process several times, all while the device is recording sounds underwater. The goal is to see if the dolphins will watch this behavior and begin to mimic one of the artificial whistles to ask for the item.

Find out more about the Wild Dolphin Project at Wild Dolphin Project

 

bnew

Veteran
Joined
Nov 1, 2015
Messages
63,087
Reputation
9,641
Daps
172,661



New AI Models Claim to ā€˜Talk to the Animals’ Like Dr. Dolittle: Here’s What You Need to Know​


Written by

Madeline Clarke

Published April 16, 2025



A dog calling someone through phonr.

Bark to the Future: AI Is Learning to Speak Animals

Artificial intelligence may be the missing piece needed to make the longstanding human dream of communicating with animals possible. Earth Species Project researchers have created NatureLM-audio, an AI audio language foundation model that can use animal vocalizations to identify various aspects of communication and behavior. NatureLM-audio is the first large audio-language model designed specifically to analyze animal sounds.

Trained on a curated dataset that includes human language and environmental sounds, the AI model can detect and identify the species of animals producing the sound, classify different types of calls, and predict the animal’s approximate life stage. NatureLM-audio has even shown potential in identifying vocalizations of species it has never encountered before.

This isn’t the first time generative AI has been applied for translation purposes. AI models have successfully translated human languages but have had more difficulty deciphering meaning from an unknown language. This makes translating animal languages trickier, especially since researchers are working with a limited understanding of how animals communicate through sound.

About the Earth Species Project​


Earth Species Project is a nonprofit focused on addressing planetary concerns. It recently secured $17 million in grants to further its work using AI to decode animal communication. The organization aims to apply its large language model to improve our understanding of non-human languages, transform our relationship with nature, enhance animal and ecological research, and support more effective animal welfare and conservation outcomes. Advocates say using AI to decode animal communication may provide a compelling case for giving animals broader legal rights.

Applying generative AI to animal communication​


In an April 1 post, ElevenLabs—creator of the speech-to-text model Scribe—announced its new AI tool, Text to Bark, suggesting it could help pet lovers enjoy similar AI-powered animal communication tools with their furry companions. The company said its AI-powered TTS model for dogs uses a new AI-powered ā€œPawdioā€ engine to support cross-species communication, turning human language into ā€œfluent barking.ā€ While the April Fool’s Day announcement was likely just for laughs, the Earth Species Project is not the only endeavor involving AI for animal communication purposes.

The nonprofit Cetacean Translation Initiative (CETI) is an interdisciplinary scientific and conservation project that applies AI to translating other-species communication. The listening project’s initial phase involves training its AI to decode the communication of sperm whales using a one-of-a-kind large-scale acoustic and behavioral dataset. With advanced machine learning and state-of-the-art robotics, CETI is working to protect the oceans and planet.
 

bnew

Veteran
Joined
Nov 1, 2015
Messages
63,087
Reputation
9,641
Daps
172,661

1/4
@aza
We’re at the edge of something profound—decoding animal communication in ways we couldn’t have imagined. NatureLM-audio is a leap toward understanding the voices of other species all through a single model. Read the paper here: NatureLM-audio: an Audio-Language Foundation Model for Bioacoustics



GcRJNXVWQAA5b_h.png


2/4
@rcadog
You and your team's work is so inspiring!!
I look forward to the translator app being on my handheld or goggles or whatever it is first.. šŸ˜‚



3/4
@tolson12
Please join Bluesky šŸ™šŸ»



4/4
@Val_Koziol
This is profound.




To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196










1/11
@earthspecies
Today, we’re introducing NatureLM-audio: the first large audio-language model tailored for understanding animal sounds. NatureLM-audio: an Audio-Language Foundation Model for Bioacoustics šŸ§µšŸ‘‡



GcMUx4XWUAE-PUx.jpg


2/11
@earthspecies
1/ Traditional ML methods in bioacoustics struggle with species-specific data, while general-purpose audio models lack deep understanding of animal vocalizations. NatureLM-audio is trained to solve a wide range of bioacoustic tasks across species—all with natural language prompts



3/11
@earthspecies
2/ Built from bioacoustic archives & enriched with speech and music data, NatureLM-audio enables zero-shot classification of animal vocalizations. Without any fine-tuning, it can classify sounds of thousands of species from birds to whales. šŸŒŽšŸŽ¶



4/11
@earthspecies
3/ On our new BEANS-Zero benchmark, NatureLM-audio outperformed existing models in detecting and classifying animal sounds.



5/11
@earthspecies
4/ NatureLM-audio can even predict species it’s never ā€œheardā€ before. The model correctly identified new species 20% of the time—a huge step forward from the random rate of 0.5%.



6/11
@earthspecies
5/ Beyond classification, NatureLM-audio excels in novel tasks for bioacoustics:
- Predicting life stages in birds (chicks, juveniles, nestlings) 🐣
- Distinguishing bird call types 🐦
- Captioning bioacoustic audio šŸŽ™ļø
- Counting zebra finch individuals in a recording 🪶



GcMWSqWWcAAYfIJ.png


7/11
@earthspecies
6/ With the development of NatureLM-audio, we aim to address some of the persistent challenges in using ML in bioacoustics. Looking ahead, we'll add new data types to support multi-modal analysis for an even richer understanding of animal communication.



8/11
@earthspecies
7/ šŸŒ As we scale NatureLM-audio, we’re committed to ethical use, preventing biases in species representation, and addressing risks like tracking endangered wildlife. With NatureLM-audio, we aim to accelerate animal communication studies with a powerfully simple foundation model



9/11
@earthspecies
8/ Dive deeper! Check out the full preprint and demo for NatureLM-audio:
Preprint: NatureLM-audio: an Audio-Language Foundation Model for Bioacoustics
Demo: NatureLM-audio: an Audio-Language Foundation Model for Bioacoustics



10/11
@RyanFM83
This is super cool! Are there any plans to integrate this into a smartphone app?



11/11
@NewWorldMan42
Very cool!




To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196
 

bnew

Veteran
Joined
Nov 1, 2015
Messages
63,087
Reputation
9,641
Daps
172,661

How AI is helping horses speak without words


04-14-2025



How AI is helping horses speak without words​


BySanjana Gajbhiye

Earth.com staff writer

For centuries, horses have stood beside humans – on farms, in sport, in therapy, and in war. They carry our weight, follow our signals, and react with subtle cues. But one thing they cannot do is speak.

Horses show discomfort through posture, tension, or the way they walk. Yet, unless you’re a trained expert, these signs are easy to miss. What if AI could give horses a voice – not in words, but through movement data?

That’s exactly what a team of researchers from Sweden is doing. Using a blend of machine learning and synthetic imagery, they’ve created an AI model that can interpret the body language of horses in 3D.

This breakthrough system is named Dessie, and it may reshape how we detect pain or illness in animals that can’t tell us where it hurts.



Why reading horses is so difficult​


Veterinarians often rely on visual cues during clinical exams. However, movements that signal distress are subtle and easy to misinterpret.

Human observation has its limits – particularly in dynamic settings like walking or trotting. Horses may offload pain to one limb, change their weight distribution, or shift their posture slightly. These changes can indicate orthopedic issues, behavioral distress, or early signs of injury.

Traditional diagnostic tools such as X-rays or MRIs show results after the damage has taken hold. Dessie aims to catch the signs earlier, by helping humans read equine body language more precisely.

The model works by transforming 2D images into 3D representations that reflect the horse’s shape, pose, and motion in real-time.

This isn’t just about visualizing a horse. It’s about interpreting a language that’s always been there – unspoken, physical, and deeply expressive.



Isolating movement patterns​


Dessie works using a special kind of AI training method called disentangled learning. In traditional models, all the information – pose, shape, background, lighting – is bundled together. That can confuse the AI, making it harder to focus on what matters: the horse.

Disentangled learning separates each feature. It puts shape in one box, pose in another, and ignores irrelevant background noise.

This makes Dessie’s 3D reconstructions not just detailed but reliable. Researchers can now isolate movement patterns without the distraction of surrounding objects or inconsistent lighting.

ā€œDessie marks the first example of disentangled learning in non-human 3D motion models,ā€ said Hedvig Kjellstrƶm, Professor in computer vision and machine learning at KTH Royal Institute of Technology.

Dessie also doesn’t need high-end cameras or markers on the horse’s body. It can work with simple video footage, using just a single camera. That opens up new possibilities for rural clinics, breeders, and researchers who might not have access to expensive imaging technology.



Recognizing how different horses move​


To train Dessie, researchers needed massive amounts of visual data. But real-world images of horses in varied poses, lighting, and breeds are hard to collect.

So, the team developed a synthetic data engine called DessiePIPE. It generates endless horse images using a 3D model and AI-generated textures, all based on real-world breed characteristics.

This synthetic approach allows researchers to teach Dessie how different horses move – without needing thousands of live animals. DessiePIPE renders horses walking, eating, rearing, or resting, with random backgrounds and lighting conditions.

The system can even generate matched image pairs that differ in just one aspect – such as shape or pose – to train the model to notice small differences.

This method not only trains Dessie to recognize subtle motion differences but also makes the system generalize better to new environments.



AI detects how horses show pain​


Pain in horses often shows up as subtle changes in gait or stance. These cues can go unnoticed unless observed by experienced clinicians. Dessie offers a new level of insight by translating these signs into 3D metrics.

Elin Hernlund, associate professor at SLU and an equine orthopedics clinician, noted that the model helps spot early warning signs.

ā€œHorses are powerful but fragile and they tell us how they are feeling by their body language. By watching their gait we can see, for example, if they are offloading pain,ā€ said Hernlund.

With Dessie, that gait can be measured and modeled precisely. The result is a digital record of posture and movement, which can be reviewed repeatedly, compared over time, or shared across clinics.

ā€œWe say we created a digital voice to help these animals break through the barrier of communication between animals and humans. To tell us what they are feeling,ā€ said Hernlund.

ā€œIt’s the smartest and highest resolution way to extract digital information from the horse’s body – even their faces, which can tell us a great deal.ā€



Detecting problems with real-world data​


Although Dessie was trained largely on synthetic data, it performs remarkably well on real-world images. The researchers fine-tuned the system using just 150 real annotated images. Even with this small set, Dessie outperformed state-of-the-art models on benchmark tasks.

In keypoint detection tasks, where the system must locate joints or features on a horse’s body, Dessie achieved higher accuracy than tools like MagicPony or Farm3D. It also predicted body shape and motion more precisely – essential for detecting problems like lameness or muscular asymmetry.

When trained with larger datasets, Dessie improved even further, beating out models that had been trained on much more data but lacked the structure provided by disentangled learning.



AI model isn’t limited to horses​


Though built for horses, Dessie isn’t limited to them. Its architecture is flexible enough to generalize to similar species like zebras, cows, or deer. The model can reconstruct these animals in 3D, despite never having been trained on them directly.

This opens the door for broader applications in animal welfare, research, and conservation. Endangered species, for instance, could be studied using just photographs and videos, without the need for intrusive monitoring.

The researchers even demonstrated Dessie’s ability to process artistic images – paintings and cartoons – and still generate accurate 3D models. That shows just how well the system can separate core features from visual distractions.

The road ahead: Limitations and ambitions​


While promising, Dessie still has limitations. It works best when there’s only one horse in the frame.

If the model encounters unusual body shapes not present in the training data, it struggles to adapt. The team hopes to solve this by incorporating a new model called VAREN, which has better shape diversity.

The experts are also expanding Dessie’s library of visual data. To do that, they’re reaching out to breeders worldwide.

ā€œTo achieve this, we’re asking breeders to send images of their breeds to capture as much variation as possible,ā€ said Hernlund.

With more diverse images, Dessie could learn to identify breed-specific traits, track genetic links to motion patterns, and improve care for horses of all types.

Letting horses speak through movement​


Dessie doesn’t teach horses a new language. Instead, it helps us finally understand the one they’ve always used. By converting motion into a digital voice, the AI makes communication between horses and humans more accurate and empathetic.

It marks a step toward a future where animals can tell us more – where their movements carry the weight of meaning, and where science helps us listen.

For horses, and maybe for other animals too, the silence might finally be over.

The study is published in arXiv.
 
Top