Meta AI’s Imagine features, which use generative AI to turn text prompts into images, are now being expanded across Facebook and Instagram, the company announced at Meta Connect 2024 on Wednesday. With the update, users will be able to use prompts to generate AI photos directly in their feed, Stories, and for their Facebook profile pictures.

The new capabilities could help users and creators call more attention to their posts and shares by allowing them to generate fanciful and eye-catching images to accompany their text. Once the images are shared, friends and followers can see them and react to them or even mimic them, Meta says.

Facebook users already upload photos of characters, animals, or something else besides their own photo as their profile picture to better protect their privacy. Now they won’t have to seek out such a photo: They can simply generate one.

The AI can also suggest captions for Stories on Facebook and Instagram, as a part of this update.

The image generation capabilities will come to Messenger, too, to create personalized chat themes. This is accessed by tapping on “Themes” in the chat. Before, users could change the background of their chats as well as the color of the text bubbles, but Meta AI offers far more options in terms of the types of images that can be used.

Meta says it’s also testing new AI-generated content in Facebook and Instagram feeds where it will display AI images created for users based on interests and trends.

This is largely designed to push people to try Meta AI by tapping on a suggested prompt to reimagine the photo or by swiping to generate content in real time using Meta Imagine AI.

Meta CEO Mark Zuckerberg remarked during the event that Meta AI differentiates itself not only by offering “state-of-the-art AI models but also unlimited access to those models for free, integrated easily into our different products and apps,” he said.

“So Meta AI is on track to being the most used AI assistant in the world by the end of this year. In fact, it’s probably already there. … We’re almost at 500 million monthly actives, and we haven’t even launched in some of the bigger countries yet,” Zuckerberg added.

Meta AI is starting to catch up with Google when it comes to AI-powered photo editing. On Wednesday, at the Meta Connect 2024 conference, the tech giant announced that Meta AI will now be able to help you edit photos using AI technology and answer questions about the photos you share.

The additional features are made possible because Meta AI is gaining multimodal capabilities, powered by its Llama 3.2 models. This means you can now share photos in your chats, not just text, similar to Google Gemini and OpenAI’s ChatGPT.

When sharing a photo, Meta AI can understand what the image contains and answer questions about the image. For example, Meta suggests you could share a photo of a flower and then ask the AI what type of flower it is. Or you could share a photo of a delicious dish and ask Meta AI how to make it. Of course, how accurately Meta AI responds to these and other questions still needs to be tested and reviewed.

Another key feature with the added photo support is the ability to edit images using AI.

After sending Meta AI the photo, you can ask it to make some sort of change — like adding or removing an object in the foreground, changing your outfit, or updating the background of the photo in some way, like adding a rainbow to the sky, for instance.

Meta AI can also be used on Instagram when you reshare a photo from your feed to your Instagram Stories. Here, the AI technology can look at the photo, understand the images, then generate an accompanying background for your Story.

Beyond photo edits, Meta is also testing translation tools for Facebook and Instagram Reels that include automatic dubbing and lip-syncing. These tests will initially be run in small groups in the U.S. and Latin America in both English and Spanish.

Other Meta AI photo features include the expansion of Meta AI’s generative AI features and the rollout of tests of Meta AI images shared to your Facebook and Instagram feeds, to prompt users to try the feature.

Fine text during Meta’s demo noted the editing features were coming to the U.S. in English first.

Noted Meta CEO Mark Zuckerberg during the event, “Meta AI differentiates itself in this category by not just offering state-of-the-art AI models, but also unlimited access to those models for free, integrated easily into our different products and apps,” he said. “So Meta AI is on track to being the most used AI assistant in the world by the end of this year. In fact, it’s probably already there. … We’re almost at 500 million monthly actives, and we haven’t even launched in some of the bigger countries yet,” Zuckerberg added.

Announced at Meta Connect 2024, Meta is launching a new VR app called Hyperscape that renders recreations of real-life spaces in high fidelity, so you can explore them while wearing a Quest headset.

Available in beta in the U.S., Hyperscape will soon let you use a phone to scan a room and then recreate it. Alternatively, you can step into a room that someone else has scanned and shared.

Meta CEO Mark Zuckerberg showed a “hyperscape” onstage of rock band Green Day’s studio.

“We’re also bringing photorealistic spaces to the metaverse, and we call this hyperscape,” Zuckerberg said. “It’s pretty wild, and we’re planning on just adding more and more over time.”

A Meta blog post explains more:

“By utilizing Gaussian Splatting, a 3D volume rendering technique, we can take advantage of cloud rendering and streaming to make these spaces viewable on a standalone Quest 3 headset,” the post reads. “In the future, creators will be able to build worlds within [our VR platform] Horizon.”

Hyperscape is compatible with the Quest 3 and Quest 3S.

Meta CEO Mark Zuckerberg announced updates to the company’s Ray-Ban Meta smart glasses at Meta Connect 2024 on Wednesday. Meta continued to make the case that smart glasses can be the next big consumer device, announcing some new AI capabilities and familiar features from smartphones coming to Ray-Ban Meta later this year.

Some of Meta’s new features include real-time AI video processing and live language translation. Other announcements — like QR code scanning, reminders, and integrations with iHeartRadio and Audible — seem to give Ray-Ban Meta users the features from their smartphones that they already know and love.

Meta says its smart glasses will soon have real-time AI video capabilities, meaning you can ask the Ray-Ban Meta glasses questions about what you’re seeing in front of you, and Meta AI will verbally answer you in real time. Currently, the Ray-Ban Meta glasses can only take a picture and describe that to you or answer questions about it, but the video upgrade should make the experience more natural, in theory at least. These multimodal features are slated to come later this year.

In a demo, users could ask Ray-Ban Meta questions about a meal they were cooking, or city scenes taking place in front of them. The real-time video capabilities mean that Meta’s AI should be able to process live action and respond in an audible way.

This is easier said than done, however, and we’ll have to see how fast and seamless the feature is in practice. We’ve seen demonstrations of these real-time AI video capabilities from Google and OpenAI, but Meta would be the first to launch such features in a consumer product.

Zuckerberg also announced live language translation for Ray-Ban Meta. English-speaking users can talk to someone speaking French, Italian, or Spanish, and their Ray-Ban Meta glasses should be able to translate what the other person is saying into their language of choice. Meta says this feature is coming later this year and will include more language later on.

The Ray-Ban Meta glasses are getting reminders, which will allow people to ask Meta AI to remind them about things they look at through the smart glasses. In a demo, a user asked their Ray-Ban Meta glasses to remember a jacket they were looking at so they could share the image with a friend later on.

Meta announced that integrations with Amazon Music, Audible, and iHeart are coming to its smart glasses. This should make it easier for people to listen to music on their streaming service of choice using the glasses’ built-in speakers.

The Ray-Ban Meta glasses will also gain the ability to scan QR codes or phone numbers from the glasses. Users can ask the glasses to scan something, and the QR code will immediately open on the person’s phone with no further action required.

The smart glasses will also be available in a range of new Transitions lenses, which respond to ultraviolet light to adjust to the brightness of the room you’re in.

Mark Zuckerberg said Wednesday during Meta Connect 2024 that Meta AI is one of the most used AI assistant across the world with “almost” 500 million monthly active users. Zuckerberg said that Meta AI is on track to become the most used assistant across the world.

“We’re almost at 500 million monthly activities, and we haven’t even launched in some of the bigger countries yet,” Zuckerberg said at the event, pointing out that it hasn’t launched in the EU yet.

Meta had previously noted that there are more than 400 million monthly users across the world. The company’s CFO, Susan Li, said during the Q2 2024 call that India is Meta AI’s biggest market, largely thanks to WhatsApp, which has more than 500 million users in the country.

In comparison, OpenAI’s ChatGPT chatbot has more than 200 million weekly users.

Meta today announced a new family of models called Llama 3.2 with new multimodal capabilities. The company is also launching Llama 3.2 1B and 3B lightweight models — models that weren’t trained on European user data  — in Europe.

At the event, Meta also announced new celebrity voices and lip-synced translations to Meta AI. The company also extended its generative AI-powered “Imagine” feature across Facebook, Instagram, and Messenger to create photos through prompts. Plus, Meta AI can understand your photos and give you answers about them and even edit them.

At the Meta Connect 2024 conference, CEO Mark Zuckerberg said that the company is developing a “neural interface” that can be used to control its prototypical Orion AR glasses.

The interface, which seems inspired by devices developed by CTRL-labs, a brain-machine interface company Meta acquired in 2019, takes the form of a wrist-worn wearable. Wearers can gesture while wearing the wristband-type peripheral to navigate around apps on the paired Orion glasses.

“[Orion is] the first device that is powered by our wrist-based neural interface,” Zuckerberg said. “[This is a] device that allows you to just send a signal from your brain to [Orion].”

The Verge’s Alex Heath reports that the wristband will soon go on sale and will work with Meta’s other AR hardware. Pricing has yet to be announced.

Meta’s Orion glasses — which remain very much a concept at this stage — are true AR. Orion utilizes tiny projectors built into the glasses’ temples to create a heads-up display.

AI has already given us some weird demos. And friends, it’s only going to get weirder from here. As the tech world runs screaming head first into the uncanny valley, lines will blur and everything will continue getting stranger.

Take, for example, Wednesday’s Meta Connect keynote. There was a constant drum beat of Meta AI and Llama demos during the event, but one in particular stood out for its strangeness.

Mark Zuckerberg invited creator Don Allen Stevenson III to join him onstage. The Meta CEO proceeded to pick up a phone and carry on a conversation with an AI-generated version of Stevenson as the genuine article stood between the exec and an image of himself on the big screen.

The demo showcased AI Studio, a platform for designing custom chatbots. The program, which rolled out to all U.S. creators in July, started with text only. Wednesday’s event, meanwhile, showcased how creators can essentially create virtual versions of themselves.

“If you’re getting a ton of DMs, and you’re missing the time to respond, Creator AI for me has been answering a lot of my frequently asked questions,” Stevenson said of how he’s used the platform. “It saves time. I have time to go outside, take a break.”

Meta ran the demo at the same presentation that gave us chatbots voiced by Awkwafina, Dame Judi Dench, John Cena, Keegan-Michael Key, and Kristen Bell. It only gets weirder from here.

Meta Connect 2024 is a developer-centric event featuring a keynote from CEO Mark Zuckerberg. He showcased new hardware and software to support two of Meta’s big ambitions: AI and the metaverse. This included everything from new Quest headsets, updates to Meta’s Llama AI model, and even a new fully AR glasses prototype.

Meta Connect 2024 kicked off live at 10 a.m. PT Wednesday, September 25. You can watch the archive via Meta Connect’s site, on Meta’s developer Facebook page, and you can still dive into Horizon Worlds from your Meta Quest headset if you really want to feel the full immersion.

We’ll be compiling all of the announcements and reveals out of Meta Connect 2024 below and will keep it updated throughout the event and after as more reactions and analysis arrive.

In a surprise announcement, Zuckerberg dramatically revealed Orion, which Meta says will eventually become the company’s “first consumer full holographic AR glasses,” though they won’t be arriving on any consumers’ faces anytime soon. Touting Orion’s light weight, compatibility for hand-tracking and eye-tracking, and most ambitiously, a neural interface, Zuckerberg also touted some impressed, and famous, early testers (including Nvidia CEO Jensen Huang). The Orion glasses were presented as the future of Meta’s AR efforts, with our own ongoing coverage of Meta Connect’s final big reveal here.

Meta’s reveal of the Quest 3S was indicated by several pre-event leaks and confirmed during Connect 2024. The Meta Quest 3S comes as a less expensive alternative to the Quest 3, starting at $299 for a 128GB version and 256GB for $399. The entirely wireless headset is compatible with the existing library of Quest apps and games, though Meta was particularly enthusiastic about its mixed-reality features. With the reveal of the Quest 3S, the Quest 3 will drop from $649 to $499. Get our early impressions of the new Quest 3S here ahead of its October 15 ship date.

Quest 2, Quest Pro discontinued

“With Quest 3S on the shelf, we’re officially winding down sales of Quest 2 and Pro. We’ll be selling our remaining headsets through the end of the year or until they’re gone, whichever comes first,” Meta told Dakidarts about the discontinuation of two of their headsets in the wake of the Quest 3S announcement.

Meta AI is finding its voice. Users can ask a question or otherwise chat with Meta AI out loud through Messenger, Facebook, WhatsApp, and Instagram and get simulated vocal responses in return. The digital voice assistant will roll out with multiple voice options to choose from, including the AI voices of celebrities like Dame Judi Dench, John Cena, Awkwafina, Kristen Bell, and Keegan-Michael Key.

The reveal comes a day after OpenAI announced the long-awaited rollout of its Advanced Voice Mode feature, with a new look and five new voices. And speaking of vocal announcements, Zuckerberg touted that Meta AI now has 500 million users.

Meta’s multilingual Llama family of models has reached version 3.2, signifying that several Llama models are now multimodal. Llama 3.2 11B and 90B can interpret charts and graphs, caption images, and pinpoint objects in pictures given a simple description.

Llama 3.2 11B and 90B might be able to look at a map of a park and tell you how long a certain path is, or when the terrain could become steeper. Or, provided a graph showing a company’s revenue over the course of a year, the models can quickly spotlight the best-performing months of the bunch.

Notably, Llama 3.2 11B and 90B can’t be accessed in Europe due to the EU’s Act restrictions. As a result, several Meta AI features available elsewhere, like image analysis, are disabled for European users.

Ray-Ban’s Meta smart glasses are getting upgrades as the company continues to try and make the case that the smart glasses can become the next big consumer device. At the forefront of the pitch is the addition of real-time AI video processing, which lets you to ask the Ray-Ban Meta glasses questions about what it can see in front of you.

The sunglasses will also be getting a smartphone-like reminder feature; live language translation between English and French, Italian, or Spanish; and full integration with music streaming apps like Amazon Music, Audible, and iHeart Radio.

Yes, we know what Zuckerberg’s shirt says

Zuckerberg’s evolution into a self-made fashionista was on full display during Meta Connect, with a shirt that spelled out: “Act Zuck aut nihil,” which for those not well versed in Latin, translates to “either Zuck or nothing.” The phrase is originally associated with Roman Emperor Julius Caesar, further cementing Zuckerberg as a Rome Guy.

In a feature that will be familiar to anyone following OpenAI, Google, and Apple’s application of image-based searches, Meta showcased its AI’s capacity to respond to prompts based on images, as well as edit photos provided to the AI based on feedback given by the user. It can also share them to Instagram Stories.

More AI in your Facebook feeds

If you’re on Facebook, you’ve probably seen a lot of AIs recently, knowingly or unknowingly. There’s an entire nascent industry around using it to generate engagement bait. Earlier this month, the label noting whether content has been edited using Meta’s tools was shrunk down. And now Facebook will actually pitch you on AI content that it generates for you that it thinks you’ll want to share or engage with.

Tests for translated, dubbed content from creators

Going a step beyond just translating captions, Meta announced a test that would use its AI translation tools to take creator content, create a translated dub, and sync the creator’s lips accordingly. So far, the experiment is limited to creator videos made in the U.S. and Latin America, with translations between English and Spanish.

In what ended up being a surreal moment, Zuckerberg brought on creator Don Allen Stevenson III for a conversation on the stage, which then turned into a demo during which Zuckerberg interviewed an AI version of Stevenson while the real deal was right beside him.

Amid the big AI reveals and hardware and announcements, Meta did dive into the broader purpose of Connect — building ties with its community of developers. As it continues to pitch developers on helping build out the Quest app store’s offerings, Meta released new support for Android apps on the Quest, along with support for 3D and spatial apps.

Batman, Wordle, and the rest of the game’s announcements

A number of new experiences were revealed for the Quest ecosystem at Meta Connect 2024, led by Batman: Arkham Shadow, which launches October 22 and will also be bundled with new Quest 3 and 3S purchases until April of next year. Beyond games with familiar concepts like Alien: Rogue Incursion for Xenomorph-filled horror and a remake of the zombie game Arizona Sunshine, Wordle of all things was announced for the Quest. The VR version of the daily puzzle will be free to play without a New York Times subscription.