1/21
@GoogleDeepMind
We’re bringing powerful AI directly onto robots with Gemini Robotics On-Device.
It’s our first vision-language-action model to help make robots faster, highly efficient, and adaptable to new tasks and environments - without needing a constant internet connection.
https://video.twimg.com/amplify_video/1937508794801487872/vid/avc1/1080x1350/TaBBdjMe2byVQ5LE.mp4
2/21
@GoogleDeepMind
What makes this new model unique?
It has the generality and dexterity of Gemini Robotics - but it can run locally on the device
It can handle a wide variety of complex, two-handed tasks out of the box
It can learn new skills with as few as 50-100 demonstrations
3/21
@GoogleDeepMind
From humanoids to industrial bi-arm robots, the model supports multiple embodiments, even though it was pre-trained on ALOHA - while following instructions from humans.
These tasks may seem easy for us but require fine motor skills, precise manipulation and more. ↓
https://video.twimg.com/amplify_video/1937509533041012737/vid/avc1/1080x1350/iFEUIT-xWqQg-m0S.mp4
4/21
@GoogleDeepMind
We're also launching the Gemini Robotics software development kit (SDK) to help developers fine-tune the model for their own applications, including by testing it in the MuJoCo physics simulator.
https://video.twimg.com/amplify_video/1937509681909407744/vid/avc1/1920x1080/CbQdg18a0ZqAm4Nr.mp4
5/21
@GoogleDeepMind
Our new on-device solution runs independent of a data network - making it optimal for applications needing speed, or situations with poor connectivity.
We’re excited to continue exploring the future of bringing AI into the physical world. Find out more →
Gemini Robotics On-Device brings AI to local robotic devices
6/21
@ccharliewu
Awesome
7/21
@LaurenceBrem
Will you open source it?
8/21
@MaxBlazh
Robots now think, see, and act locally.
9/21
@KevinFi69594692
@yacineMTB
10/21
@____Dirt____
Question: Can we bang it?
11/21
@AlvigodOP
Technology truly is the ultimate for peace to the world
12/21
@sageadvik
bc ek baar baith ke poora bna le, har baar kya "put the apple in the basket" ??
13/21
@_fracapuano
@NepYope
14/21
@Prashant_1722
This is incredible right into the future. Google is on a roll this year. Loving it. Gemini
15/21
@MickeySteamboat
16/21
@Caaasy
awesome
17/21
@Kuper_xx
Epico
18/21
@turing_hamster
can it solve a rubik’s cube
19/21
@BensenHsu
Breakdown of the paper behind it:
Title: Gemini Robotics: Bringing AI into the Physical World
The study addresses the challenge of bringing advanced artificial intelligence, particularly large multimodal models that excel in digital tasks, into the physical world to control robots. While these models show impressive general abilities in areas like understanding text and images, making robots truly useful requires them to understand and interact with the physical world competently and safely. This involves what the paper calls "embodied reasoning," which is the common sense humans have about 3D environments, object relationships, and basic physics. Current robots often lack this deep understanding, limiting their ability to perform complex, general tasks.
...
20/21
@TimeLoopx
About time. Still playing catch up in research?
21/21
@purepathwill
Impressive.
These on-device models put Gemini Robotics firmly on track to become the 'Android of Robotics'.
In the limit, OEMs will just need to focus on building the best robotics hardware, and simply use Gemini for the 'brain'.
To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196
