What Is the Role of an AI Designer?

How AI designers are bridging the gap between user needs and technological capabilities.

About 4 months ago, I began managing the product design organization for Facebook’s Artificial Intelligence team. We are a central organization that provides AI services for Facebook, Instagram, and other Facebook apps. We also work to develop new experiences, powered by AI.

Before joining the Artificial Intelligence team I had been reading a lot about AI and was really excited about the idea of working in an emerging space. I’d been working on mature mobile and web apps for the last 10 years, so being in a position of learning and ambiguity felt really energizing. I also felt a responsibility knowing that AI technology will be extremely transformative to society, more-so than even the mobile revolution has been. I wanted to use my skills to ensure that AI tools are built in a responsible way.

To be honest though, I wasn’t exactly sure what the role of product design would be on the AI team. We don’t own a consumer product surface of our own, so there isn’t an app for the team to maintain day-to-day. I also had a preconceived bias against designing “technology first” versus “people problem first”, so figuring out how to work effectively in a world where technological capabilities are unfolding real-time (and you are working to apply them in the right way) felt backward and unfamiliar to me.

I thought it might be useful to others to share the approach we are taking to offering design support to our developer & application team partners, and in the creation of new experiences. This information might be useful if you are also building out an AI design team, or if you are curious about pursuing a role designing AI.

Generally speaking, designers on the team are working on the following project types:

Designing AI Prototypes

When an AI tech team needs a prototype, demo, or visualization of their technology, they work with a product designer. AI designers make sure that people see what’s possible with AI. We create prototypes showing how a particular technical capability might be used by people when the AI is working well. As an example, we might create a demo of AI suggesting a possible caption for your Instagram post, or AI helping you know where to buy a pair of shoes you see in a post.

For an AI Demo designer, the creation process is usually to illustrate a set of product ideas as one page briefs with a set of visuals to briefly illustrate the idea. We use the one pager to get buy-in from the broader team that the idea is worth building out further. The finished prototype be used to green light a new research initiative or development project within the AI org.

Designers working on AI Demos are working on a wide variety of ideas, they are highly generative rather than going deep in one technology area. They have a strong understanding of current AI capabilities and keep track of how new capabilities are developing. As a designer working on demos, you might be creating a visual search experience, on a way that users with visual impairment can “see” newsfeed posts by rolling over the photos with their fingers.

Shaping New Technology

AI designers can also work as an embedded part of a technology team, shaping new technology. AI designers work on long-term areas of investment like computer vision, speech, language, video, and AI assistant initiatives. These project teams are developing new AI capabilities (rather than working with existing AI capabilities). The work you do as a designer on a technology team blurs the lines between design, PM and research. You are working to ensure that developer teams understand user needs relating to a particular type of technology and are building the technology to directly solve those needs. You are imagining user experiences that might be seen by users in 3–5 years, and would take significant improvement in AI technology to achieve. AI designers go deep in understanding the long-term vision for AI in their area of focus.

Developing AI Centered Products

Another project an AI designer might work on is to design new AI centered applications, testing them for viability, and (if promising) pitching them for further investment. The time horizon for projects in this space are farther out, so you are working in a world where you imagine people living life 5–10 years from now. You are thinking about how AI can offer better socially powered services to people longer-term. The products you build might be the Facebook or Instagram of the future, or might be a totally new way of engaging with your community.

Designers in this space often have a background in building new hardware or working in a startup space on new app ideas. They are comfortable with ambiguity and have an understanding of how to create the criteria for continuing to pursue or shut down a line of exploration.

Collecting Data for AI to Learn

One of the most important parts of shaping AI technology is giving AI the right data to learn. AI designers work with the engineers who build tools for this data collection and annotation, and design the platforms that enable it, to streamline efficiency and make it intuitive to collect good quality data. In some cases designers help gather datasets when more automated methods won’t work.

Here is an example of how a designer might help with data collection helps AI learn: We may need to teach AI to know when a person is raising their hand. In order to teach the AI, we gather thousands of video examples of people raising their hand, ensuring that examples come from people of various ages, genders, ethnicities and physical differences so that the data set is inclusive. They then gather thousands of examples of people NOT raising their hand to help the AI know when the answer to, “Is the person raising their hand,” should be, “No.” Finally, they show examples of people raising their hand that are potentially harmful (i.e. someone raising their hand in a Nazi salute) so that the AI can learn to flag those examples when identified.

AI designers help to ensure that we are developing data collection systems that are safe, fair, ethical, and apply to real user problems. These designers create frameworks and guidelines where needed to ensure the safe and secure use of AI. They design the libraries that our machine learning systems use to learn, ensuring that they are free of bias, clean and effective.

Designing AI Developer Tools

Designers on the AI team also build applications used by AI engineers, as well as the external developer community through open-source projects. Product designers on this team are building specialized developer applications, ensuring that we are creating a set of tools that work well as a system. Designers in this space need to have a deep understanding of the AI development process and a passion for empowering engineers across the globe. They often have a strong background in engineering or systems thinking.

At a high level, AI designers are bridging the gap between user needs and technological capabilities. Knowing that our goal is to give people the power to build community and bring the world closer together, we want to use current and future capabilities of AI to design the future of how people connect and collaborate.

There are big differences in the role of a typical product designer and an AI designer. Rather than launching a product feature that shows up in an app in an immediate and obvious way, our output is often clarity for engineers on how the technology could be applied. Because AI capabilities might take 2–3 years to develop, it’s important for designers to help developers understand the potential of different solutions and their impact on people’s lives when developing AI. And even more exciting — Beyond using existing AI capabilities to solve problems that exist on our platforms today, these designers get to imagine and imagining how AI will reshape the world tomorrow.