Mark Zuckerberg Owned Meta Launched a New AI Model ‘ImageBind’

On Tuesday, Mark Zuckerberg-owned Meta launched a new AI model called ImageBind

Meta Launched a New AI Model: Similar to Google and Microsoft, Meta (formerly known as Facebook) is going all out on artificial intelligence (AI). On Tuesday, Mark Zuckerberg unveiled the testing and open-sourcing of their AI models. In the latest development, the company has announced a new open-source AI model, called Meta ImageBind, that combines different senses – six to be precise – to create experiences.

Talking about the AI model, Mark Zuckerberg said, "Today we're open-sourcing ImageBind, a new AI model that combines different senses just like people do. It understands images, video, audio, depth, thermal, and spatial movement. Check out the video for some examples of what it can do now, and I'm looking forward to seeing what you all build with it."

How does Meta ImageBind Work?

A research project at this point, the project can use generative AI to create immersive, multisensory experiences. By using image-paired data, ImageBind can learn a single joint embedding space for multiple modalities, allowing them to "talk" to each other and find links without being observed together. This enables other models to understand new modalities without resource-intensive training.

"ImageBind equips machines with a holistic understanding that connects objects in a photo with how they will sound, their 3D shape, how warm or cold they are, and how they move," the company said.

For example, if you give the model an image of a tiger and audio of a waterfall, it combines this input data to make a video with both elements. If you give a model input like "small creature" (text), "rainforest" (image), "rain" (audio), and a photo of a bird (IMU), it will combine these to give a video.

As per the company's statement, "ImageBind is part of Meta's efforts to create multimodal AI systems that learn from all possible types of data around them. As the number of modalities increases, ImageBind opens the floodgates for researchers to try to develop new, holistic systems, such as combining 3D and IMU sensors to design or experience immersive, virtual worlds."

Meta said that ImageBind could also provide a way to explore memories — searching for pictures, videos, audio files, or text messages using a combination of text, audio, and image.