EarthRanger in augmented reality

One of the best times of year at the Allen Institute for AI (Ai2) is the annual hackathon competition. Each year, members of our organization pitch their wildest ideas and join together in small teams. We work against the clock over the course of three days to come up with the best, hackiest solution to a big problem, or work that furthers our mission of creating artificial intelligence (AI) for the common good. It's a chance to get to know team members outside our usual circles and work on an energizing project. This past year, as a part of the hackathon competition, I pitched an idea to the group, and want to share the resulting work with you.

We wanted to bring the work of Ai2's EarthRanger team to a broader audience by creating an engaging experience that could be understood by anyone, regardless of their previous conservation or AI experience. We envisioned this augmented reality (AR) experience could be used at a conference booth, or could travel around in an educational context to universities, other potential EarthRanger partners, environmental journalists, or AI research institutions.

We were also motivated to create a design-focused hackathon team to experiment with design methods outside of our usual workflows. Working with AR as a medium allowed us to explore an immersive experience and challenged us to think about storytelling in mixed reality.

One of our primary goals was to highlight EarthRanger's partners. At EarthRanger, our partners are our greatest resource and we believe conservation software is only as impactful as the partners who deploy it. It was meaningful to spend this hackathon presenting the Twiga Tracker initiative, which is led and supported by two EarthRanger partners: the Giraffe Conservation Foundation (GCF) and the Smithsonian Conservation Biology Institute (SCBI), respectively. GCF is the only NGO in the world that focuses solely on giraffe conservation and management in the wild throughout Africa. Their organization's initiatives span across 18 African countries and concern all four species of giraffe.

GCF's Twiga Tracker Initiative uses GPS devices to conduct the largest tracking study ever conducted on giraffes. With well over 300 giraffes tracked to date, GCF and their research partners use EarthRanger to manage and visualize the data, leading them to make informed, conservation-related operational decisions.

Our first challenge was to align on the goal of this project and the narrative we wanted to tell. Since this team had never worked together before, I wasn't sure how difficult this part would be. However, we were able to agree on some goals and parameters with relative ease, giving us more time to work on the prototype.

Our prototype included several elements, and we relied on the diverse skills of our team to achieve each part. One element that was particularly unique to this work was the physical panorama, which would serve as the context over which we laid the AR elements. In order to make this physical representation as relevant as possible, communications manager Caitlin Wittlif researched the actual location where GCF had deployed tracking units in Murchison Falls National Park in Uganda. By researching where the giraffes were translocated and looking up a Google Street View of the roads where conservationists would most likely drive, we were able to build out a physical panorama based on the reality of the landscape to ground the entire AR experience.

We also simultaneously started reading about the work the Giraffe Conservation Foundation is doing. This helped us to shape the story we wanted to tell. We began to pull together footage we had from GCF as well as images, maps, data points, and quotations. From there, software engineer Kate Panter started to write our narrative script.

With the script in development, it was time to begin thinking about the flow of the prototype interactions. We created a flow diagram to help us visualize the user interactions through the prototype. Starting with the user's entry point into the prototype, we then discussed the timing and assets needed as the experience progressed. We were able to pinpoint areas of potential confusion and imagine the user's experience better through this exercise.

With these pieces coming together, the final missing element was the AR assets. Our team members, senior technical artist Eli VanderBilt and UX designer Jenna James worked to find 3D giraffe assets. Because of the short timeline, we tried to reduce the amount of custom assets we created in 3D. Even so, Eli put in a tremendous amount of work posing each giraffe and creating custom animations to visualize the problem of giraffe population decline and the deployment of GPS tracking devices.

On the last morning of the hackathon, before we were scheduled to present our work, it was time to tie up loose ends and present to the group for hackathon judging.

We have already seen exciting outcomes from this work. This hackathon team was particularly successful at building working relationships between team members who typically don't work together. Cassidy Trier, who usually works as a product designer on Ai2's Semantic Scholar, has continued on to do other work with the EarthRanger and Skylight teams at Ai2.

This AR prototype also inspired discussion on Ai2's PRIOR team. Because working with Adobe Aero performed so impressively, the team decided to seek out more iOS tools for converting real-life home interiors into 3D, leading them to expand their workflow to include more tools.

Since this is only a prototype, we still have some work to do before this project could be released to the public or expanded upon. Our proposed next steps include:

• Refine the story by working with partners (GCF)
• Conduct usability testing on this prototype
• Increase accessibility by adding captioning and audio descriptions of visual elements
• Develop the prototype in code
• Expand to more EarthRanger partners

Our team was made up of five members from across Ai2's projects: Caitlin Wittlif (Communications Manager at Ai2), Cassidy Trier (Product Designer at Semantic Scholar), Eli VanderBilt (Senior Technical Artist at PRIOR), Jenna James (UX Designer 2 at Applied Science and Technology), and Kate Panter (Software Engineer at ReViz). We credit this project's success to the broad skill set and hard work this team brought to the table. We also had input from Jordan Steward, Ai2's Conservation Communications Manager.