In the previous post, I effectively followed the breadcrumbs scattered by the posing of this question: What effect does the haunting, background white noise of expert systems have on our attention?
One definition for expert systems is: those non-local touchpoints of human problem-solving that mediate our modern existence whether we like it or not (e.g., car insurance network—and the full spread of its tentacles).
They are in the background in that they are not systems that we directly interface with on a daily basis—I get in my car daily. I don’t think about insurance. But it’s there—haunting the background. If I get in an accident, there’s a number to call. Somewhere. An automated system. A representative. A claims process. I don’t actively recall how it works, but I know it’s there. Waiting. Dormant but activated.
This awareness—faint, peripheral, constant—is exhausting in a way I can’t quite name. It’s not that I’m worried about insurance. It’s that the potential interface is always humming at the edge of consciousness. A portal I might have to open. Someday. Maybe.
Yet, they are also haunting because despite being distant, they are brought near to me just enough that if I listen closely I can hear their white noise present all around me—I sense, however subtly and quickly, that if I were to get into an accident, I could dial a customer service number somewhere, find some way to navigate the annoyance of automated messaging routes, talk to an agent somewhere, and resolve my mess situation somehow.
The key thought is that this is different than in previous eras of human history. Sure, there have always been problems to solve, things that could impact me from outside my local context (e.g., a regime change), and a tacit awareness of this that comes and goes—sometimes, and for some people, more explicit than others. However, expert systems are so vast, and present for many problems that previous would have been solved though local, personal relationships. If we pause to reflect on it, we are surrounded by portals to far-off lands systems to solve our problems impersonally.
Charles Taylor’s model of a difference between a porous self (open directly to the agents of nature) and a buffered self (sealed off from the agents of nature through intermediary expert systems) captures this dynamic. However, we should be careful not to interpret the buffered self as being closed off to any external agents. The buffered self is still porous, just porous to the intermediary expert systems—hence we can be haunted through their portals.
Putting it all together, if we constantly have to tune out the haunting white-noise of abstract systems that are often stuck in the borderlands between background and foreground, then does this create a habit of more easily drowning out any minutiae that seems peripheral?
Oddly, to ask this question sends me through the portal to encounter a distant-yet-embedded answer (or material for framing a potential response): “The cause of inattention is due to ADHD, which can have biological and environmental factors…blurp blurp trauma…blurp blurp prefrontal cortex.” But what if this is just more white noise, providing another opportunity to habituate my tuning-out since the haunting jargon is vague, impersonal, and not providing satisfactory answers?
The clear irony is that expert knowledge about ADHD rushes from the borderlands to the foreground whenever I contemplate my actual experience with overlooking matters of minutiae and try to make sense of it—but what if the expert knowledge emanating from the portal is the very cause of my habit of overlooking the minutiae in the first place?
In all of this, my musing is to reason through the potential phenomenon created by modern expert systems that lurk in the borderlands between the foreground and background—between haunt and portal.
Now, what if we followed the breadcrumbs toward another potential side effect of expert systems—namely, the shattering effect emanating from an expert system failing us?
In The Consequences of Modernity, Anthony Giddens identifies that one effect of our interfacing with expert systems is that it replaces a methodical, rational process for each of our contingency plans with a vague trust in expert systems as a whole.
Now, offloading tasks to others with expertise is nothing new. If I broke a wheel on my horse and carriage, then I might consult a local expert to conduct the repair. At most, I might have the postal service ride off with a letter to deliver it to a contact in another state. But even then, I am still contacting someone I personally know and trust (or have been recommended to by someone I know and trust).
The difference is that modern expert systems are like a portal in which I transport from the personal and familiar to the distant and abstract to mediate my problems that previously where personal and imminent.
Moreover, these modern expert systems are enclosed—its not only that I don’t know the point of contact there personally, its that somewhere along the pipeline I hit a black box. I cannot penetrate expert systems (easily), but they can haunt me.
The effect of this transition from personal and imminent relations to the abstract and distant systems is more radical than it may initially seem.
When I interface with the personal and the imminent, I have no black box. I can quite easily take in all the circumstances and make a conscious decision:
The wheel of my carriage broke. I visit a local craftsman. If it’s too late, I stay at an inn, or I hope that a local is willing to host. The craftsman appears trustworthy and capable of fixing my wheel. He charges me X. I can afford X. I accept that the local craftsman fixes my wheel.
Now, when I interface with an expert system, I am much more likely to deal with a black box. I cannot easily take in all the circumstances and make a conscious decision:
My wheel is broken. Assuming I don’t have a spare that I know how to fix, I call AAA. I listen to an automated message as I wait to talk to someone, somewhere. Someone answers. A tow truck is sent my way, driven by someone from somewhere. Turns out, I have to wait for a repair. I need a ride home. I request an Uber driven by someone coming from somewhere. The next morning, I call insurance to order a rental. They tell me I only have coverage for X…and on and on it goes.
Notice that I am now no longer taking the course of actions I am out of personal trust and preference, I follow the systems that are just there. I don’t think about it much, but I guess I trust them—I know someone, somewhere likely uses them. It all feels a bit blurry, cold—but I have a solution that feels direct.
Now, some interactions can still remain personal and imminent. You might be the type of person that appreciates knowing the person who is going to do work for you if needed. You might build rapport and trust with a local mechanic. However, you don’t have to be that person, many people don’t know how to be, and you can’t always be that person. At some point, you depend upon and/or interface with an expert system that is a black box to you.
So, what do you need? You still need trust that someone can help you—but it doesn’t have to be personal and imminent. Often, it is blurry, vague—and tacit—trust.
Now, unless someone is quite conscious and committed about this (rare), then this vague trust is universalized. I don’t have the time in a modern context to research every expert system I interface, or could interface with. And so:
I guess I trust Geico.
I guess I trust Kroger.
I guess I trust Jeep.
I guess I trust AWS.
I guess I trust Chase.
I guess I trust Apple.
I guess I trust McDonalds.
I guess I don’t have much of a choice. If I were to map out the full set of risks in these interactions, or some sort of alternative, I would be overwhelmed.
So, I guess I will trust—all of it.
But even this requires a level of consciousness that most do not have. We operate from blurry, tacit trust toward the expert systems that haunt us.
Now, what happens if one of these expert systems breaks my trust?
On the surface, this might seem insignificant. If I am disappointed by Jeep, I will go with Ford. If I am concerned about AWS, I will select Microsoft. If I don’t like Kroger’s produce, I might consider the other grocery store in town.
But the truth is more haunting. Because the trust wasn’t really in Jeep or Kroger or AWS specifically. The trust was in the system itself—the background hum that things work, that someone somewhere has this handled, that I can (fully) enter the portal when needed.
When Jeep fails me—not just a bad product, but a betrayal (a recall they hid, a warranty they denied, a customer service agent who lied)—something deeper fractures. Not just “I won’t buy Jeep again.” But: Can I trust Ford? Can I trust any car manufacturer? Can I trust the regulatory agencies that approved this? Can I trust the mechanics who service these vehicles?
The glass shatters and spreads. One black box proved untrustworthy. But I was trusting all the black boxes with the same tacit, universal faith.
If entering one portal proves damaging, then what do I do with all these portals, all around me and haunting me?
But here’s the difficult: If the haunting is tacit, as well as the vague, universal trust in expert systems, then would I even recognize the tacit paranoia that lingers and spreads?
Likely, if this really started to manifest as “clinical anxiety,” then I would be referred to a therapist. Someone, somewhere that apparently got a degree by studying under someone, somewhere. Someone trusted this therapist somewhere and somehow. They must know how to treat my anxiety, somehow because they developed the skill to do so somewhere.
I will divulge myself, and they will meet me with therapies and techniques that came about somehow from some expertise from somewhere. Oh, I look forward to them curing my anxiety somehow, in some way.
Does the therapist realize that they are introducing more jargon, more uncertainty, more vagueness, and, overall, more of that haunting blur?
I highly doubt it. They learned therapeutic techniques from someone, somewhere. They trust the research that came from somewhere, somehow. They’re just as haunted by portals as we are.
So the shattering effect cascades in two directions:
One system fails and distrust spreads to all systems (Jeep → Ford → regulators → mechanics → everything). Because the distrust was tacit, never overt, it isn't at all apparent. There might be an intensification of the haunt, but it was always there and haunting before. This intensification cascades, and anxiety that comes from it drives us to seek help from a new portal, a new black box, and a new haunt.
The point: We have not worked out the potential recursive loop that expert systems can create. The expert systems that create our anxiety offer to cure it through more expert systems. The shattering spreads tacitly. The anxiety remains tacit. The attempted cure adds more portals to haunt us.
And because it’s all tacit—the trust, the shattering, the anxiety, the paranoia—we can’t even name what’s happening. We just know something feels wrong. The world feels brittle. We feel fragile. We seek help, and the help makes us feel... not better, exactly. Just more managed. More mediated. More dependent on systems we can’t see and don’t fully trust.
That’s the shattering effect.
Perhaps there was something to it when CS Lewis wrote:
I like bats much better than bureaucrats. I live in the Managerial Age, in a world of “Admin.” The greatest evil is not now done in those sordid “dens of crime” that Dickens loved to paint. It is not done even in concentration camps and labour camps. In those we see its final result. But it is conceived and ordered (moved, seconded, carried, and minuted) in clean, carpeted, warmed, and well-lighted offices, by quiet men with white collars and cut fingernails and smooth-shaven cheeks who do not need to raise their voice. Hence, naturally enough, my symbol for Hell is something like the bureaucracy of a police state or the offices of a thoroughly nasty business concern. —Original Preface to the Screwtape Letters