April 2023

AI2 Newsletter
Screen grabs from the Phone2Proc app, including examples of a phone scan of a room and real world deployment.

Top story

Phone2Proc: Bringing Robust Robots Into Our Chaotic World
Training embodied agents in simulation has become mainstream for the embodied AI community. However, these agents often struggle when deployed in the physical world due to their inability to generalize to real-world environments. AI2's PRIOR team has improved this process with a new embodied AI training approach: Phone2Proc. This three-step process allows you to train for successful, robust agent behavior in a real environment.

Chatbots and AI Literacy
With OpenAI's ChatGPT and Bing's new chatbot, AI models are being centered in news coverage and making people more aware of artificial intelligence's technical strides. But there is also plenty of misunderstanding surrounding this coverage, which brings the spotlight back to the importance of AI literacy. In a piece written jointly by AI & Society lead Nicole DeCario and AI2's interim CEO Peter Clark, the two identify five tips for engaging with chatbots responsibly.

Objaverse: A Universe of Annotated 3D Objects
A virtual room filled with 3D objects from Objaverse.
3D datasets have historically been hard to come by, making it difficult to train robots to understand real-world environments. AI2's PRIOR team has responded to this challenge with Objaverse, a gigantic and diverse database of 3D models. As Devin Coldewey from TechCrunch explains, "Objaverse includes variations on common objects so the model can learn what defines them despite their differences."

A Computer Vision System That Writes Code
In a recent effort, the PRIOR team has created a computer vision system called VisProg that solves complex compositional tasks described in natural language by generating and executing programs. Each line of the generated program may invoke one of several off-the-shelf computer vision models, image processing routines, or python functions to produce intermediate outputs that may be consumed by subsequent parts of the program. In other words, VisProg solves tasks by writing code!
Semantic Scholar logo

Semantic Scholar's TLDR blog post explains how the team measures the impact of TLDRs to assess their helpfulness to users! Learn more ➞
A photograph of Tara Wilkins, a woman with long brown hair wearing a necklace and earrings, standing in front of a blue wall.

Featured AI2er: Tara Wilkins


IT engineer Tara Wilkins is our Featured AI2er for April! Learn about her experiences in IT, what she's working on at AI2, meet her adorable cat Lucy, and more on the blog.

Read more ➞
The NLP Highlights podcast logo

NLP Highlights Ep. 139


In this episode of NLP Highlights, Kevin Yang describes their story generation system, as well as the challenges involved in evaluating long generated texts.

Listen Here ➞

More from us
➞ These tools help visually impaired scientists read data and journals
➞ What do our machines dream of?
➞ Sonia Fereidooni Awarded the 2023 Allen AI Outstanding Engineer Scholarship
Visit us: allenai.org | Follow us: @allen_ai | Work with us!

AI2 Color Logo