Discussing which jobs will be replaced around the campfire

Last fall, I went camping with several other families from my neighborhood. As we sat around the fire the conversation turned from Sam Altman’s ouster at OpenAI to what the AI revolution will mean for us more generally. We had lawyers, professors, therapists, architects, designers, teachers and parents in the conversation. While there was a lot of speculation about what humanity may lose in the AI revolution, one concern has stuck with me.

One professor of therapy insisted that there was one thing an AI model could never replace. Even though a large language model like ChatGPT could be trained to respond like a therapist, adhering to best practices and responding with empathic sounding words, an essential part of the healing that takes place between therapist and client is the experience the client has of being seen, their story being known and understood by another person. No matter how good the AI is, it cannot provide this.

An essential part of the healing that takes place between a therapist and client is the experience the client has of being seen, their story being known and understood by another person.

Humans evolved for connection

I think this is a critical observation. Human beings are wired for interpersonal connection. We evolved and succeeded in becoming the most powerful species on the planet by cooperating in groups. We devote a tremendous amount of brain power to simulating what other people around us are thinking so we can be better family members, colleagues and teammates. And yet this aspect of human minds also tends to make us attribute agency to things that are not in fact agents. Neuroscientist, Justin Barrett, in his book, Born Believers, explains the evolutionary advantage to the person who attributes the rustling they hear in the bushes to the lion waiting to pounce instead of simply assuming it’s the wind –and being wrong.

And we know from years of research that humans are likely to attribute agency and personhood to software especially when interacting with them via verbal exchanges in text.  A good deal of Chat GPT’s attractiveness as a product can be attributed to its effectiveness at responding like a person would by using large samples of human responses as its source material.

It’s obviously unethical to try to create experiences that deceive users into believing a person is interacting with them when in fact there is only software. But is it possible that users informed that they are in fact talking to software only, could still experience the positive psychological impacts of feeling seen and known simply on the quality of the responses of the software, having been trained on how humans communicate those sentiments?

Can AI make people feel seen?

In my past work at Twin Health, as we experimented with large language models, we constantly kept in mind that at the core of our member’s trust in our product is trust in the people who care about them and are paying attention to their health in a way that, often, no one really had before. Our members’ trust in the program and in the behavior and medication changes we were asking them to make is based on their trust primarily in people, not primarily their trust in their Digital Twin or AI.

This doesn’t mean companies should shy away from using these tools especially where they can unburden our human care teams from tedious repetitive work or provide real-time decision support for people in ways that on human could afford to. But it means that one key metric we should keep our eye on is how seen, cared for and in relationship our users feel as we increase automation to enable the scale that unlocks new levels of care to match to the size of the problem of chronic diseases like diabetes.

Over and Under thinking

On my walk this morning I was talking to my friend and linguist, Luke Wakefield, and I had an insight. For most of my life I’ve been deeply curious about how things work: everything from electronics to religion. I make models of the world to best fit the data of what I know and when I get new data I can’t really rest until I’ve expanded the model to make sense of it.

Most of the people I live and work with have come to understand this as part of my process. But I also regularly hear the phrases: “you’re overthinking it.” This always takes me by surprise as it usually comes when I feel like I’m getting close to actually understanding and articulating the dynamics of a thing from it’s underlying animating principles.

My “ah hah!” moment this morning came from the realization that what they may be expressing is that this thing I’m talking about, from their experience or way of thinking, comes easily, intuitively or automatically without the kind of conscious explanation or thought I’m articulating.

Which is to say, that it comes to them from what Daniel Kanneman calls system 1 thinking: a rapid, automatic, unconscious kind of thinking that runs in the background and gives us all kinds of useful predictions, intuitions and perceptions. This is contrast to system 2 thinking which we all tend to use when our system 1 thinking fails to make sense and we have to switch over to conscious, high cognitive load, high attention interrogation of what is going on here. Most of us can drive to and from work automatically, staying entirely in system 1 thinking. But if we have to negotiate a tricky merge in traffic or drive to a new location and find street parking? We’re likely to switch over to system 2.

The human brain even has dedicated functional areas or “modules” if you will, that specialize or can be trained to specialize and create automation or automaticity in certain tasks like: facial recognition, encoding and decoding writing, and more. This is pretty obvious from experience but what’s less obvious is that there is wide variation in the human population in the distribution and relative effectiveness of these modules.

Brain module diversity or disability?

Don’t have the module for facial recognition? We call that prosopagnosia or faceblindness. It doesn’t mean you can’t recognize faces. It just means that for you, it becomes a conscious, cognitive effort you have to spend energy, time and attention on, while for everyone else who has the module, it’s just a service their brain provides at no cost. Or in Kanneman’s framework, without the brain module to create the automation, the task moves from system 1 thinking to system 2 thinking.

Don’t have the module that does rapid automatic naming of written words? We call that dyslexia. It doesn’t mean you can’t read. It just means that it will cost you more. You’ll work twice as hard as people who have the module for half the speed and accuracy. And you’ll probably develop some hacks to get around needing to read with that part of your brain except when absolutely necessary.

The human brain is amazing. Even when the more common functional area doesn’t work, it can create new automations over time from practice and use. Or it can develop adaptive enhancements of other modules that do work well. My wife and 2 of my kids are faceblind. And they recognize voices faster than faces, which is especially fun when watching animated films. And while my two dyslexic daughters read more slowly than their classmates, they can run circles around most people in processing what’s being said and beating you to the punchline or the next point in the argument.

System 2 is Slower, but it can show its work

Another interesting consequence of having to develop conscious processes of doing the kind of processing that, for many, is unconscious and automatic, is that people who use system 2 thinking have more direct conscious access to the principles, values, and strategies that animate these actions. This means we can often put words to and describe them with more specificity and detail. But when we try to share this with someone for whom it’s simply an automatic and unconscious service their brain provides, it probably does sound like we’re overthinking it. While from the other perspective, their way of thinking and reasoning about it is:
1. not universally available to everyone

2. literally under thinking in that it’s thinking taking place below the conscious mind and inaccessible to it.

I see this dynamic operating among some autistic people who sometimes don’t have the kind of brain automation that can make perceiving and learning some unspoken social and cultural rules unconscious and intuitive. So instead they can become functional anthropologists, observing and studying in order to derive these complex rules and dynamics through observation. It’s no surprise then when, through this more conscious access to their thought process they are able to notice and describe bias, unfairness, and inconsistency in the social and cultural norms that are built up largely through unconscious system 1 thinking driving interactions between individuals and groups. By slowing down into System 2 thinking, we gain access to new insight we don’t have when we rely mostly on our brain’s System 1 auto-pilot.

Diverse neurologies enable a richer reality

The relatively new public discussion happening online and on social media about neurodiversity has done a great job of spreading a shared public vocabulary around attention, executive function, sensory differences. But what I’m hopeful about is the developing awareness of brain difference not universally as deficit from a default norm, but as normal variation in the population of a social species who is made stronger and richer by differences in ways of thikning and being in the world.

So the next time you hear someone talking in detail about something that to you seems obvious and automatic, before you accuse them of overthinking, pause and consider the opportunity to hear from someone who may have had to spend more energy to bootstrap their own brain function in this area from first principles rather than simply take what their unconscious mind has served up to them without effort.

Or, how remote work can trick you into feeling more stressed than you need to

There is no trash can or recycling bin in your computer, just different ways the system tags files. But the useful fiction of the recycle bin interface lets us know what to expect about files we put there even if it’s not literally true. In a similar way, we might think of emotions as a kind of software interface to what might otherwise be an overwhelming amount of sensory and analysis data. The work our emotions are doing is a kind of summary of sensory inputs, our perceptions about them, and their matching to prior experiences.

As useful as these emotion-interfaces are, it’s important to remember that they aren’t reality itself any more than the recycle bin icon is, and to keep a critical eye on what they have to tell us about reality.

New Job: new anxiety?

I recently started a new job. I was feeling some stress about it as I learned about my new teammates and worked to pick up projects mid-stream. In the early mornings when I would wake up, I would feel a familiar, burning acid stomach feeling. I’ve always felt anxiety, dread, and fear there in my stomach and my half-awake mind doesn’t have the benefit of all the things I know when fully awake.

One day, when trying to get back to sleep, I propped up on some pillows, elevating my torso. To my surprise, the sensation I had been reading as anxiety stopped immediately. I was having a sensory experience that my mind was reading and tagging as anxiety, but by changing my position, the sensory input stopped and the emotion quickly evaporated. Sometimes these interfaces, heuristics, and strategies we use to know what we are experiencing get it wrong or at least don’t get the whole picture.

Extended Senses; extended emotions

If emotions are a kind of interface to the overwhelming flow of sensory inputs and perceptual judgments our minds make of them, then it follows that our senses are some of the primary inputs for our emotions. This is interesting to note when considering the current discussions of AI. Everyone seems preoccupied with the question of their sentience but no one seems to talk about how whatever sensors we give them or they find a way to get will dramatically impact whatever analog they develop for emotions or consciousness. Another way to say this is that an octopus has a different consciousness than a mammal precisely because its bodily and sensory inputs are different.

We live at a time when we have dramatically extended our sensory inputs from people and objects in our immediate vicinity to a whole host of people and systems across the planet. I have joined three different remote companies over the last 5 years. Joining each one was not so much agreeing to be at a particular place at a particular time as much as it was agreeing to hook a set of notifications up to my consciousness and engage them with a certain level of throughput.

Each time I do this I notice a distinct uptick in my anxiety, stress, and sense of overwhelm– at least until I gain enough institutional knowledge to filter the signal from the noise. Which is precisely a process that our brains do with new sensory inputs as well. At first, a new environment can feel too noisy, too bright, or smells too intense or distracting. But quickly most people’s brains filter out sensory inputs that are consistent and non-threatening. That said, difficulty doing this characterizes much neurodivergent sensory experience. Not everyone’s brains do this for them automatically.

Alarms, Alerts, and Notifications As Senses

I’ve long been scrupulous as a designer and a user about when and how I let systems interrupt my attention. I believe our attention is our most precious and scarce asset. And, once diverted, getting my attention back focused where I want it is costly. Further, these interruptions are most often un-designed or underdesigned and at worst, exploitative. Why would I invite some random app and the design and product teams behind it to hijack my attention multiple times a day?

After living with near-constant awareness of my daughter’s blood glucose and diabetes health through continuous glucose monitoring, I can tell you that I have come to feel like an additional sense. This makes a lot of sense given the work of neuroscientist, David Eagleman, on sensory augmentation and substitution.

The internet allows us to tie new senses into our minds. Is it any wonder that they are contributing to new and sometimes negative emotional outcomes?

I’ve written elsewhere about the ways that I enjoy feeling my senses extended by technology. Knowing that my daughter’s glucose is in a safe range while she and my wife are asleep sleep halfway around the world while I travel is invaluable. I love being surprised by my phone’s voice assistant reading texts from family and friends to me that arrive while I’m out on a morning walk and feeling closer to them than ever as their thoughts seem to unfold in my mind as read aloud by my voice assistant in my Bluetooth earbuds.

What gives me pause is the need to bring that same critical eye to the emotional interfaces my mind brings to these new sensory inputs. If my brain can misread signals from my stomach as anxiety, it’s probably going to misread signals from my work Slack too and roll them up into some kind of emotional experience that may or may not be a real picture of reality.

As a member of the community of design practitioners, I think we have a lot to learn from this metaphor of notifications as extended senses. How might we design notifications that don’t demand full attention hijack from our user with adrenaline-infused audio alerts, but instead follow how our senses work with more subtle dial-ups and dial-downs of attention through more senses than the visual and the auditory? I’ve begun to explore this a bit in my bgAWARE project but there’s a lot more work to be done to move away from the current all-or-nothing paradigm that’s fracturing our attention constantly.

So? What helps?

If you haven’t already, I highly recommend spending some time auditing and managing which apps and input streams are allowed to interrupt you and when. This is arguably one of the most crucial mental health interventions you can make. Slack has robust controls over when it’s allowed to ask for your attention. iOS and Android have rolled out new tools to silence, group and delay notifications. These are worth learning about and using.

As far as positive practices, the most helpful ones I’ve found for bringing this critical eye to these emotional experiences are, writing, meditation, and emotion logging.

Writing

Making time for personal writing, journaling and reflection often results in new perspectives and re-frames on emotions that at first seem very reliably tagged. Kevin Kelly says he writes in order to know what he thinks. This description of making time to write captures so much of what I find valuable in the practice. It gives me space to explore what I’ve been feeling and thinking in a way that I can’t do alone with my thoughts because I simply can’t sting enough of them together before they start falling out of my attention. I’ve been using the practice of morning pages: making time to sit down and write every day without agenda, without goal. I type and I don’t target a word count or a number of pages. I try to write for 20 minutes. I don’t always make time for it but I notice that when I do, I feel less anxious, more present, and more able to be the person I aspire to be to the people to whom I’m committed.

Meditation

Meditation practice builds the habit of an inner observer or executive function watching the river of thoughts and emotions roll by, driven by the current of sensory input. Meditation does this by cutting down on the signal input or limiting and focusing sensory attention.

Emotion Logging

Finally, I’ve been striving to learn and use more names for my emotions. The psychology literature is clear that the more and different kinds of emotions we learn to perceive and name in ourselves, the healthier we will be. And this makes intuitive sense given this metaphor of emotions as interfaces. We are literally giving our minds more and more nuanced interface elements with which to build emotional interfaces to summarize and understand our experience. I built an app called CharacterMe focused on helping teens understand and name their emotions. Lately, I’ve been loving the award-winning app, How We Feel, for support in taking time to attend to, name, and log my emotions.

What’s the point?

This is not an essay against Slack or email or notifications. Although I have critiques of how all 3 could work better with what we know about our senses, our attention and how they impact our emotions.

This is a call to:

1. Pay attention to what new senses we link to our consciousness

2. Remain curious and skeptical about the emotions that come along with them.

Our brains do a lot of work for us automatically and below the level of our conscious selves. But some of the emotions are as fictional as the recycle bin on your computer’s desktop. And I’ve found that my well-being is rewarded by being skeptical about these interfaces and checking my brain’s work on a regular basis.