Category: Cognition

What can people do better than machines? The view from 1951

Image from Paul Fitts' paper on huma machine differences

What can humans do that machines can’t? The pessimistic view, in a world of advancing AI and robotics, would be “less and less every day”. While researching this today I ran into an interesting historical perspective — the view from 1951.

It’s a paper called “Human Engineering for an Effective Air Navigation and Traffic Control System,” written for the National Security Council by Paul Fitts, a psychologist known for studying human factors in technology. He was most famous for his work trying to figure out how to automate air travel to make it more safe, and that’s where this report comes in. It’s a long, elegantly-written attempt to tease out the distinctions: Which aspects of airflight could best be managed by machines, and what by humans?

The fun begins on page 6, where Fitts asks …

Callout from Paul Fitt's paper: what can man do better than machines

He argues that our “sensory functions” and “perceptual abilities” are more sensitive than machines — which is arguably less true today than it was back then, given innovations in everything from LIDAR and 3D imaging to olfaction. He also argues we meatfolk have the edge on “reasoning” — which, again, is certainly true but probably much less true than in 1951, when it was considered pretty rad to get a vacuum-tube computer to play a simple peg game.

But then Fitts points out a unique human cognitive advantage: “Flexibility”.Flexibility. Another special Capacity of the human is his extraordinary flexibility and ability to improvise. These abilities are still in completely understood by psychologists, but they represent important respects in which humans surpass machines. The amount of flexibility a machine has is fixed by the amount that was built into it. The machine will attempt as many different kinds of solutions as it's designer plan for and no more. Experiments on complex problem-solving and humans, on the other hand, show that humans may attempt many different solutions to the same problem – just think of the number of ways in which this paragraph could've been written to convey essentially the same point. Flexibility is especially important in the changing and evolving system, such as one in which new techniques are constantly coming into use. It also provides insurance against complete breakdown in emergencies. The conclusion here is that if flexibility in the system is important, it probably is a good plan to let human beings play an important role in the system.

Now, this distinction seems still very solid! AI and robotics are remarkably more advanced than in 1951, but even the most advanced machine is still pretty narrow in its function. “General” AI is still nowhere in sight, and robots are still simply terrible at navigating and manipulating the everyday human environment: Google’s Go-playing computer creamed the world’s champion at that game, but it couldn’t pick up the game pieces by itself. In any individual intellectual and physical skill, you could create (or soon create) a machine that would best us, probably. But human ability to adapt to new situations and tasks makes us the cognitive Swiss Army knife in this corner of the universe.

(We could extend Fitt’s list by adding a bunch of areas in which humans still trounce machines, by the way — like empathy and creativity. Fitts didn’t include these, possibly because he didn’t see them as related to airflight control, though I could make the case they’re necessary in that field.)

This point about flexibility reminds me of Robert Heinlein’s dictum:

A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects.

If I’d written that list I’d probably have included a bit less fighting and invasion-planning, but otherwise it’s a pretty solid CV. Of those twenty-one skills, though, I’m only confident of my ability in fifteen.

Go download that whole paper by Fitts — it’s a fun read! (That awesome image at the top of this post is from the paper, BTW.)

Share

The odd pleasures of reading Proust on a mobile phone

The first page of Proust's "Remembrance of Things Past" on an iphone

A year ago I wrote an essay about reading War and Peace on my iPhone. If you missed it, why hey! You can read the whole essay here!

When people found out what I was doing, they often claimed it was aesthetically bonkers. Isn’t it uncomfortable and weird, they wondered, to read such a massive book in tiny screenfuls? You can’t fit many words on a single screen. Indeed, I tend to blow my font-size up pretty big, so there’s barely one or two hundred words on each Kindle page.

But the thing is, this made the experience oddly retro. Visually, a Kindle screen on a mobile phone harkens back to … the early days of novels.

Back in the 18th and 19th century, people often read novels that were printed in the teensy “octavo” format. It made books extremely portable, so they were, in way, much like the iphones of the premodern period: Pocketable culture. Here’s a page from Conjectures on Original Composition, a book from 1759 by the English poet Edward Young:

Page from Conjectures on Original Composition, a book from 1759 by the English poet Edward Young:

Looks rather like a iphone Kindle screen, doesn’t it?

At any rate, today I discovered an essay by Sarah Boxer describing her even more Olympic feat of novel-phone-reading: The entire 1.2-million text of Proust’s À la recherche du temps perdu. (That’s the first page, above, on my phone.)

Boxer did an even better job of describing the peculiar aesthetic delights of reading a ginormous book screen by screen. Let me quote it here at length:

Soon you will see that the smallness of your cellphone (my screen was about two by three inches) and the length of Proust’s sentences are not the shocking mismatch you might think. Your cellphone screen is like a tiny glass-bottomed boat moving slowly over a vast and glowing ocean of words in the night. There is no shore. There is nothing beyond the words in front of you. It’s a voyage for one in the nighttime. Pure romance.

In a curious way, I think reading Proust on your cellphone brings out the fathomless something in the novel that Shattuck calls “the most oceanic—and the least read” of 20th-century classics. It makes you feel like Jules Verne’s Captain Nemo in his submarine, which is just right. As Benjamin Taylor notes in his biography, Proust: The Search, this is how Jean Cocteau described the writer at work in his bedroom, the cork-lined retreat on Boulevard Haussmann that Proust called “a little bottle stop” muffling the sounds of the world.

Although Proust knew exactly where he was heading when he put together his masterwork—he began with the first and last parts, then turned to the middle—the same cannot be said for his readers, no matter how they tackle his text. They are at sea. This is what makes reading the novel such hard going, particularly in the middle. It is also what makes the experience extraordinary.

Knowing where you are, physically, in a bound book keeps you from feeling this oceanic feeling quite so much. It keeps you grounded. But reading the book on your cellphone emphasizes your own smallness, your at-sea-ness, in relation to the vast ocean. There you are, moving along without any compass. How brave you are in your little dinghy, adrift and amazed.

That’s such a great metaphor: The book as an ocean, the e-reader as a tiny porthole!

It reminds me of the way I look at paintings in galleries: I zoom in as close as I can, so I can examine the tiny individual brushstrokes in as much detail as possible.

I once read a piece by an artist noting that painters experience their painting in two modes — up super-close, and from far back. A lot of the time they’re hunched over the canvas, going stroke by stroke. So, as the artist pointed out, if you want to see what that experience was like, you want to get as close as physically possible to the completed painting, and study it from barely an inch or two away. (As you can imagine, shoving my nose right up to a canvas does not make me super popular amongst art-gallery guards; I’ve nearly been tazed at the Art Gallery of Ontario, trying to get myself with microns of Tom Thomson’s The West Wind.) But as the artist went on to note in his article, painters also frequently stand back ten or twenty feet to appreciate the overall scope of what they’re doing — so you too ought to zoom out often as you’re absorbing a painting. The painter’s experience of their own painting is simultaneously a) brushstroke by brushstroke and b) twenty feet away.

It occurred to me once, while nose-close with a painting, that novels (and other forms of longform writing) have a bit of the same dual-focus aspect: The writer composes word by word, sentence by sentence — but also has the entire text in mind. We readers experience the whole book both as a single bolus of culture and a collection of individual thrilling sentences or passages.

Boxer’s lovely metaphor comes the closest I’ve seen to evoking that literary duality.

Share

On the cognitive value of having a bazillion tabs open on your browser

my open tabs

Right now, I have 52 brower tabs open. Behold the madness above! (The tabs are off to the far right; it only shows the top 21.)

It’s kind of nuts. Why do I work like this? Is there any good reason to keep open so many tabs?

“Tab freaks” like me are in the minority. When the Mozilla foundation did a study of Firefox users, they looked at the high end of people’s tab usage, i.e., what was their maximum number of tabs they kept open. The biggest chunk of people clustered around 5 or so tabs — people with 20 open tabs were pretty uncommon. Nutjobs like me, with 52 tabs open, are so far out on the X axis we don’t even appear on the chart, heh.

chart showing how many tabs people keep open in Mozilla Firefox

Why precisely do we “power users” keep open so many tabs? Me, I do it partly as a memory aid. As I’m reading and researching, I’ll find a link to something interesting, and open the tab in the background so I can read it later, when I’ve got time free. Or, in the middle of a piece of reading, I’ll get an idea I want to chase down — so I’ll briefly interrupt myself to a) google the concept, b) find a relevant document, and c) open it in the background so I can, again, come back to it later.

How typical is my “memory” behavior? Pretty common, it appears. A while ago I read this fascinating 2010 research: “A Study of Tabbed Browsing Among Mozilla Firefox Users”, where the researcher Patrick Duboy and his coauthor interviewed 21 browser-users to ask about their tab usage. Many of the subjects talked about using tabs to manage remembering things:

  • “Having the tab open is a reminder to me. […] if it’s at the end of the day or lunch time while I am cracking a sandwich or something and I’ll say, ‘Oh yeah, I want to go back and look at that link’ because I see the tab sitting there.”
  • “If I search for something on Google, I just go ‘right-click, rick-click, right-click’—you know, opening all the tabs. And then I would look at them later.”
  • “Usually I find interesting links halfway through the article, and that’s why I like tabs.”
  • “Just a quick little side task, when, you know, I want to come right back to what I was working on.”

I often leave tabs open for days, which again turned out to be a reasonably common behavior:

  • “If it is something I plan to refer back to in the near future, like later that day or the next day, […] sometimes I’ll leave the tab open.”
  • “I will often lose interest in something, and I think I might go back to it, so I will leave the tab open and open a new one. I don’t necessarily always go back to them, but the opportunity is there if I want to.”

One interesting finding: Of all the tabs people opened, 25% were never looked at. People opened them to leave dangling the possibility that they might look at them … but they got around to it before closing the entire browser window.

It also turned out that people flipped between tabs pretty frequently, using them to A/B compare stories or pieces of information. In fact, this was such a common behavior that 71% of the time someone flipped over to a tab, it was a revisitation.

When I look at all this data, and reflect on my own behavior, it reminds me a lot of the ergonomics of paper — and specifically, “pilers” of paper.

A black and white old photo of a man looking at a pile of paper on his desk

Back in the 80s, MIT’s Tom Malone (not the person in the picture above, BTW, that’s just some rando) studied people’s desks and how they dealt with paper documents. He found two types of people: The first group were “filers”, people who filed every document after working with it, because they liked having a clean desk. Then there were “pilers”, who had stuff all over the place and were quite happy with it. You might think that the pilers are just disorganized freaks who never get anything done, but that’s not what the research showed; indeed, as Abigail Sellen and Richard Harper found in The Myth of the Paperless Office, “pilers” could usually find a document just as fast, or faster, than someone who’d filed it neatly away.

What’s more, the riot of documents in front of “pilers” could be creatively useful. In a terrific 2002 article on “The Social Life of Paper”, Malcolm Gladwell summed this up nicely:

But why do we pile documents instead of filing them? Because piles represent the process of active, ongoing thinking. The psychologist Alison Kidd, whose research Sellen and Harper refer to extensively, argues that “knowledge workers” use the physical space of the desktop to hold “ideas which they cannot yet categorize or even decide how they might use.” The messy desk is not necessarily a sign of disorganization. It may be a sign of complexity: those who deal with many unresolved ideas simultaneously cannot sort and file the papers on their desks, because they haven’t yet sorted and filed the ideas in their head. Kidd writes that many of the people she talked to use the papers on their desks as contextual cues to “recover a complex set of threads without difficulty and delay” when they come in on a Monday morning, or after their work has been interrupted by a phone call. What we see when we look at the piles on our desks is, in a sense, the contents of our brains.

In a way, this is what I do with browser tabs. I let them pile up — sometimes for days! But later on I can wind up re-glancing at them and having a memory or idea triggered. Sure, the tabs are chaos, but they can be creative chaos.

I say “can” be, because frankly the ergonomics of browsers are still pretty lousy. Paper strewn around a desk — and an office — has good glanceability. You can often tell what a paper document is with a very quick glimpse. Book spines work the same way: When I pass by my bookshelves a quick glance at a spine reminds me of the contents of a book, which can trigger memories and new ideas.

color coded books on a shelf

Not actually *my* bookshelf. Very pretty, though!

(This is why I recoiled in total horror at the Marie Kondo insistence that one should toss out nearly every book you own, keeping only 30 of them. I have about 1,000 books around my house — mostly nonfiction and poetry — and getting rid of even one feels like I’m lobotomizing myself. I wrote a column for Wired a few months ago detailing my freakedoutness at the Kondo concept.)

But browser tabs aren’t as glanceable as paper, I find. For the last year I’ve been using the Vivaldi browser (which you can see in the screenshot above) in part because it has superb tab-management features, including a control-E quicksearch that lets you refind a tab and zip over to it. That’s great for refinding, but not for idly rummaging your mind’s eye over your open tabs. Frankly, I’ve never seen anyone produce a good browser plug-in that lets one visualize open tabs nicely. I think it’s a problem of the comparatively tiny size of a computer screen. Even a 27-inch monitor is pretty chintzy compared to the real estate of a desk, where you can really spread out documents and look at them. (I seriously doubt I want to work all day long with a VR helmet on, but I was intrigued by a point that Kevin Kelly made in his piece on VR — which is that truly high-rez VR could allow you to view lots of different digital documents, spread out before you, in the way we spread out paper.)

One problem with having tons of tabs open as a memory device is that it can create huge psychological “sunk costs”: I resist closing the browser because, well, hey, there might be something valuable in an as-yet-unviewed tab, and I’ll LOSE IT FOREVER.

But while talking about tab madness on Twitter the other day, Sue Gardner pointed me to One Tab, a supercool browser plugin: When I have 53 tabs open, a single button-press closes them saves them as a single list. Which you can later open up! So now I’ve been using this thing like mad, because it adds a new mnemonic heuristic: I can look at a big huge list of days and days of links, getting jolts of serendipity and remembrance.

Indeed, I don’t keep a diary, but sometimes I think tab history is just as good better. It’s a record of where your mind has been.

Share