Brilliant Labs’s Frame glasses serve as multimodal AI assistant

Brilliant Labs, the developer of AI glasses, has launched Frame, the world’s first glasses featuring an integrated multimodal AI assistant.

This leap in wearable technology is coupled with significant support from AR pioneer and Niantic CEO, John Hanke, whose company made Pokémon Go. Brilliant Labs has raised $6 million to date.

Those riches from Pokémon Go are coming back to help the AI glasses industry.

The roster of advisors and investors now includes Coho VC, Brendan Iribe (cofounder of Oculus), Adam Cheyer (cofounder of Siri), Eric Migicovsky (founder of Pebble), Steve Sarowitz (founder of Paylocity and chairman of Wayfarer Studios), Nirav Patel (former core team member at Oculus and founder of Framework), Francisco Tolmasky (member of the original iPhone team), Plug and Play Ventures, and Moveon Technologies.

Under the leadership of ex-Apple executive and Brilliant Labs CEO, Bobak Tavangar, Frame introduced what it calls a paradigm shift in daily living. Equipped with an always-on AI assistant named Noa, akin to Iron Man’s Jarvis, Frame not only transforms how we perceive the world but also redefines our interactions with the physical environment, the company said.

“Frame truly offers a new paradigm in the way humans interface with the digital world, and with each other,” Tavangar said in a statement. “Noa, a totally new kind of AI virtual assistant, empowers users to engage more with the world around them, not less. Our hope is that Frame enables people to connect with each other in entirely new ways, and opens up new experiences in everyday life, the workplace, the classroom, and more. This is the unique promise of bringing generative AI in front of the human eye.”

Powered by an integrated multimodal generative AI system capable of running GPT4, Stability AI, and the Whisper AI model simultaneously, Noa performs real-world visual processing, novel image generation, and real-time speech recognition and translation. Frame’s multimodality empowers multiple AI systems to collaborate, unlocking limitless applications for this technology in everyday life.

Frame is not just a technological marvel but a fashion statement. It pays homage to historical figures like John Lennon, Steve Jobs, and Gandhi, who shaped culture while donning iconic circular spectacles. Frame seamlessly integrates advanced technology into a lightweight, stylish package, virtually indistinguishable from conventional eyeglasses.

There isn’t any augmented reality feature per se. Rather, Frame is designed as AI-embedded glasses, where the output can be seen and/or the glasses can be used as a real-world input for the LLMs. Normally right now people will type or upload content to ChatGPT for example to actually interface with it. Now with LLMs on your glasses, you can prompt it through real world inputs (like what you see, what you hear, and where you are in any given moment in time).

Multimodal AI Assistant With Personality

Beneath Frame’s sleek exterior lies Noa, a one-of-a-kind always-on AI assistant, the company said. Noa evolves and adopts a unique personality by analyzing the compounded interactions between the user and the AI assistant through a privacy-preserving knowledge graph.

Upon activation, users are introduced to Noa as a digital “egg” ready to hatch, reminiscent of a virtual pet. By answering a series of questions, Noa’s avatar grows into a character that reflects the user’s personality and the persona Noa will adopt. This personality becomes more apparent as Noa learns and adapts to both the user and the tasks it receives, Brilliant Labs said.

Noa’s capabilities extend to summarizing book pages and translating foreign text to structuring whiteboard sessions for colleagues. Frame’s built-in multimodal generative AI allows it to orchestrate multiple AI systems for advanced tasks like visually and acoustically measuring emotional sentiment and facial cues.

Beyond personal assistance, the company also said Frame redefines the concept of a search engine in the age of AI-enhanced glasses. Noa can instantly provide price checks by scanning clothing articles in retail stores or identify a home’s listing by glancing at it during house hunting. The possibilities are vast, promising to enhance human interactions and experiences.

Designed to be indistinguishable from eyeglasses

Weighing only 39 grams, Frame is manufactured using engineering-grade nylon plastic for comfort and durability. Presented in classic vintage styling with color options including Smokey Black, Matte Cool Gray, and a transparent variant named H2O, Frame is meant to fit with the style of any wardrobe.

The device features a high-resolution vivid display with 3,000 nits of brightness, a front-facing spatial camera, an integrated microphone, and a 6DoF IMU for advanced functionality.

Prescription lenses, provided in partnership with AddOptics, can be seamlessly bonded to Frame through a unique optical lens casting process. Frame will be available for pre-order in March 2024 at a retail price of $300.

Continuing the open source model for developers

The company’s previous product, Monocle, sold out entirely of its first orders. Now, Brilliant Labs will focus on the sales and availability of Frame for both consumers and developers.

Brilliant Labs said it is committed to open-source devices, as Frame follows the footsteps of its predecessor, Monocle. Developers can explore and modify both Frame’s hardware and software, with full resources, including living documentation, open-source codebase, and hardware schematics, made available by Brilliant Labs. The parameters of Noa’s supported AI models can also be adjusted by developers.

“Brilliant Labs’ groundbreaking progress, starting with Monocle and now with the launch of Frame, has been inspiring,” said said Adam Cheyer, cofounder of Siri and investor in Brilliant Labs, in a statement. “With a design that is both beautiful and practical for everyday use, Brilliant Labs’ Frame is the first step in ushering in a new era that’ll transform the fabric of how society operates. Frame can enhance our human capacity to organize, access and act on information with the help of its embedded AI assistant Noa.  Through its extensible and open AI platform, Brilliant Labs could introduce a new wave of hands-free applications that’s reminiscent of the beginning of the iPhone era,”

Brilliant Labs, founded in 2019 by ex-Apple employee Tavangar, along with cofounders Raj Nakarja and Ben Heald. The company has three employees.

As for video, Frame can record video and take pictures. But it is primarily for the purpose of using the multimodal AI feature as opposed to it being like Spectacles by Snapchat. For example, if you want to remix a friend’s likeness, you can look in the direction of him/her and snap a photo and command how Frame should remix his likeness.

Frame (via Noa) will tap into GPT4 and Stable Diffusion by Stability AI to create a new version of that shot. And then the image after it’s outputted (which you can see through Frame), will be wiped clean after use — Frame doesn’t store any images/videos like Snapchat or Meta might. So the purpose of the video or photo is more for the AI functionality – less for social media use.