Summary: ChatGPT’s web browsing too shallow to impress | Apple’s insistence on swallowing subscription revenue causes dark patterns in innocent 3rd party apps | AI in companies is an organizational challenge, not just a technology issue | Elon Musk is uncannily like me, according to his new biography | After 3 months, my email newsletter has subscribers from 101 countries | Scientists who investigate fraudulent research are under attack, but you can help them | Test your understanding of the state of UX in 2023: how bad are things?
This week’s feature image is by Ideogram — still the best (only!) generative AI for including words in an image. DALL-E 3 will supposedly challenge this status soon. Can’t wait.
ChatGPT Web Browsing Too Shallow to Impress
ChatGPT now has real-time access to the web, which seems like a significant advance in AI. Things do move fast this year! I am less impressed with the actual implementation. Here are two examples I tried:
(Yes, I ran some vanity questions. At least I know the correct answers with 100% confidence.)
To start at the end, it’s fake news that Rolf is my brother. I don’t know how in the world ChatGPT could have made that up. He’s a good friend and was a great colleague during the initial work on heuristic evaluation, but clearly has a different last name. Most importantly, our common ancestor was probably some Viking a thousand years ago. My real brother is a prominent Internet publishing executive in Denmark who has received plenty of news coverage, some of it mentioning our family relationship, so the information is on the web and has probably been scraped by ChatGPT in the past.
The question about my new website can be scored as maybe 50% correct. It’s reasonably impressive that ChatGPT knows about my email newsletter, even though it’s only 3 months old. But the correct answer to the question would have been uxtigers.com since that’s my website (as opposed to my newsletter).
Finally, the first of my sample questions shows the deeper problem with ChatGPT’s web browsing prowess. It does list 5 trends (as opposed to articles, as requested), but are they the most important? How can an AI not mention AI, undoubtedly the number one UX change — not just this year, but since the launch of the Web in 1991 and the GUI web browser in 1993? Pursuing the footnote reference reveals that ChatGPT’s answer is based on the conclusions of a single article. Most likely, the top-ranked hit in a web search.
AI is faster and better than search at answering users’ questions because it shortcuts the need to peruse lists of search hits. Instead of evaluating the SERP listing to determine where to click and then having to synthesize the answer from whatever partial information is found at each search hit, the AI has already read the entire Internet and will compose an answer to the user’s exact question.
That’s how AI is supposed to work and is how ChatGPT worked in the research study I linked. But the new web-browsing ChatGPT seems to visit the top search hit to summarize that one source as the answer. This approach will suffice for factual questions, such as who is the lead star in a new movie. But for questions requiring judgment, AI needs to synthesize the answer across multiple sources while judging the credibility of each. ChatGPT 4 has been surprisingly good at this in the past, which is why it’s so disappointing that it fails to provide the same level of insight for new information.
ChatGPT needs a reboot in its approach to web-enabled answering. For complex inquiries requiring layered responses, the current version is grossly inadequate.
I am underwhelmed by ChatGPT’s shallow web browsing feature. (Emoji by Midjourney.)
Poisoned Apple Induces Dark Design in 3rd Party Apps
Why is it impossible to unsubscribe from Audible within its mobile app on iPhone? (Users need to visit the Audible website in a browser to manage the subscription that’s otherwise used on the phone.)
I was getting ready to blast Audible for this dark pattern, which is surely worth about 3 skulls on my 5-skull dark-pattern scale:
3 skulls of 5 for the dark pattern of being required to move to a different device to unsubscribe. But this unpleasant design is probably not Audible’s fault, so I won’t actually award them these skulls. I am just displaying the skulls to remind you of the severity of dark patterns.
By comparison, I awarded Amazon.com 4 skulls for one of the dark design patterns they are being sued for by the Federal Trade Commission (FTC).
Luckily, I read the comments to Heath Kamp’s post about Audible’s bad iPhone unsubscribe process. Ahmet Bektes pointed out that Audible has a laudable subscription management process on Android phones, with a simple link for “Cancel membership” as the third option under “Manage Membership.” Can’t be much simpler, and I wouldn’t want cancelation to be the first option since it’s an infrequently used command.
We can entertain the possibility that Audible’s Android designer is an angel, whereas their iOS designer is a devil. But I doubt this explanation. We must dig deeper to discover why the design is virtuous on Android and evil on iOS.
The comments ride to my rescue once more: Anthony Silvia highlighted that Apple confiscates 30% of the revenues an app earns through an in-app subscription process. A great example of how business models become part of the total user experience because this punitive revenue cut prevents many companies from supporting subscriptions within the app and makes them move everything to a website.
I am not privy to Audible’s internal decision process, nor do I know whether Apple is considering allowing apps to include an unsubscribe option without confiscating their earnings. However, Silvia’s explanation seems very plausible.
Thus, I’ll refrain from criticizing Audible and instead say that the blame for the dark design pattern likely rests with the poisoned Apple:
Apple’s confiscatory approach to subscription revenue earned by 3rd-party apps on its phones is likely the underlying cause of unpleasant dark design patterns in those apps, making it impossible for users to unsubscribe. (“Poisoned Apple” by Midjourney.)
Apple’s greed doesn’t just deliver poisoned apples to the iPhone app vendors. It also poisons the well for everybody else. As users learn that it’s excruciating and time-consuming to unsubscribe, they become less likely to sign up for subscriptions in the first place. This hurts even vendors with exemplary unsub design.
Design guideline: If you’re an ethical company allowing customers to escape their subscriptions without undue pain, you should say so. For example, my email newsletter offers a one-click unsubscribe link at the bottom of every email. (If you want to be literal, it’s two clicks: one click from the email to the unsub page and a second click on that page to confirm the unsub request. I still count this as a one-click design, which is ethical and recommended. The confirmation step insures against the common fat-finger problem of mistakenly tapping a link while reading the email on a mobile device.)
The AI Paradox: High Interest, Low Implementation = Your Chance to Lead
My prize for the best posting of September goes to Paul Roetzer for his analysis of the lack of organizational leadership in embracing the AI revolution due to limited knowledge and the resulting limited visionary thinking. (There was tough competition for my award last month, which saw many great posts.)
I encourage you to read Roetzer’s complete analysis, but here’s a summary of his thinking:
While interest in AI is burgeoning, actual implementation lags dismally behind. Even among global enterprises, there is a surprising lack of urgency to address the extensive organizational changes that AI demands. Most large enterprises view AI mainly as a technological challenge. This complacency results from leadership’s limited understanding of AI's vast reach, creating a leadership vacuum in AI literacy. The slow pace of adoption offers an extraordinary opportunity for those willing to understand and apply AI, setting the stage for unprecedented career and business advancements.
The Jakob Nielsen prize for the best posting of September 2023 is hereby awarded to Paul Roetzer. (Gold Medal by Leonardo.)
Similarities between Elon Musk and Jakob Nielsen
I am reading the new biography of Elon Musk, the founder of Tesla and SpaceX and a cofounder of PayPal and OpenAI. It’s a great read with many captivating stories from the last 25 years in technology. It’s uncanny to find several similarities in the book between Musk and myself:
Favorite science fiction book: The Moon Is a Harsh Mistress
Favorite computer game: Civilization
Arrived in Silicon Valley in 1994: I as a Distinguished Engineer and Musk as an intern (he soon accelerated his career past mine)
Obsessed with design details
When hiring, prefer IQ, talent, and attitude over resumé credentialism and skills
Of course, there are also many differences. I don’t want to go to Mars; my Dad was nice. (The book makes too much hay of Musk being driven to greatness because his father was abusive. Based on personal experience, it is possible to raise successful children without abusing them.)
My main criticism of the book is that it spends too much time in the later chapter on endless details from Musk’s acquisition of X (formerly Twitter). The most interesting of Musk’s companies are, in order, SpaceX, Tesla, PayPal, Neuralink, X, and The Boring Company — the latter living up to its name.
The biggest takeaway lesson is “Musk’s Algorithm,” which is his approach to simplifying products by eliminating requirements and deleting design elements. In fact, the book has many quotes from various design meetings, where his comment was, “Delete, delete, delete.” I particularly like his statement that if you did not later have to add 10% of the deleted elements back again, you did not remove enough overhead in the first place. An example is when a rocket exploded due to an item removed before launch. It was reintroduced for subsequent launches, which went well. The exploding rocket may seem like a failure. Still, because so many other elements had been removed, the rocket’s total weight was reduced, allowing for more payload and making SpaceX the best launch company on the planet. One explosion is worth future success.
In UX, we have the advantage that our design failures become apparent in user testing rather than exploding in the market. Try out this advice and start removing things from your UI. Unless testing shows the need to add 10% back, you probably didn’t delete enough. Less Is More.
Robert A. Heinlein’s “The Moon Is a Harsh Mistress” was my favorite science book growing up. Guess what? It was Elon Musk’s, too. The plot centers around a self-aware supercomputer in a hint of what some people fear from AI today, though “Mike” is a hero in the book. (Moon Settlers by Leonardo.)
“Jakob Nielsen on UX” Subscribers in 101 Countries
Only 3 months after I started my new email newsletter, it lit up inboxes in 101 countries. It's like a canvas painted with the colors of the world! This rapid scaling underscores the global appetite for UX insights.
It is one of the most gratifying aspects of my career to watch the growing impact of the ideas I have been fighting for, starting from a narrow base in Copenhagen, Denmark, in 1983 when that entire country only had 3 usability jobs (one at each of the two biggest software developers, with each UX guy supporting thousands of developers, plus a human factors effort at the nuclear energy research center).
Global impact, unlocked! My email newsletter has worldwide reach, only 3 months after I launched the Jakob2.0 reboot. (World map by Midjourney.)
Defend Science, Fight the Replication Crisis
As you may know, there’s a significant replication crisis in several research fields, most notably social science and psychology, where many published papers contain false conclusions that do not hold up when other researchers try to replicate the studies. These problems are not always caused by fraud; statistics are so complex that it’s easy to make mistakes, like the ones I discussed recently in my article about the replication crisis in website analytics.
A small group of scientists run a website called Data Colada, pinpointing papers with suspect data that likely indicates erroneous results. (See Wall Street Journal article about this project to clean up science.) They have a strong track record of making research journals retract papers with erroneous findings, which is as it should be. They coined the term “p-hacking” for the practice of looking at more and more hypotheses until you find one that’s “statistically significant” (though it isn’t when resulting from going hunting in the data). Science corrects itself as long as future researchers can question past results and get findings retracted that do not replicate.
Unfortunately, one Harvard professor got so upset about having her papers questioned that she turned to the legal system to sue the scientists who saw signs of data tampering in her papers. (Read more about the case. She also sued Harvard University for conducting an independent investigation that agreed with the critics.) The right to question research results is extremely important for the future of science and for any hope of overcoming the replication crisis and establishing credible research practices. All of us should support the scientists who are now under attack in the American legal system, where it’s notoriously expensive to defend yourself. About 3,000 independent scientists, now including me, have banded together in a fundraiser to collect the necessary money for the defense.
An evil scientist’s lab, as imagined by Midjourney. However, most research results that fail to replicate are caused by mistakes or sloppy research methods, not by actual fraud.
UX Angst of 2023: Test Your Understanding
Here’s a short quiz about my recent article, UX Angst of 2023. The answers will be in next week’s news roundup email.
1) What is the article's stance on the future of UX as it relates to AI?
A) AI will lead to the decline of UX.
B) AI will have no significant impact on UX.
C) AI will facilitate a UX renaissance.
D) AI will replace the need for UX designers.
2) According to the article, what caused the 11% drop in UX earnings for 2023?
A) General economic recession.
B) Outsourcing of UX jobs.
C) Market correction.
D) A decline in demand for UX skills.
3) What is the attitude recommended by Jakob Nielsen to handle the challenges in the UX field?
A) Cynicism
B) Angst
C) Optimism
D) Indifference
4) What is Saffo's Law, as mentioned in the article?
A) Never mistake a clear view for a short distance.
B) The tech industry is in a permanent recession.
C) Salaries in UX are unsustainable.
D) The future of UX is bleak.
5) What does the article suggest about the long-term UX salary trends?
A) Salaries have been inconsistent.
B) Salaries have increased considerably.
C) Salaries have remained stable after adjusting for inflation.
D) Salaries will see exponential growth in the future.
6) According to the article, what proportion of companies have reached a high level of UX maturity?
A) 10%
B) 50%
C) 1%
D) 25%
7) What does Jakob Nielsen indicate about the UX world's size since he started his career?
A) It's 3,000 times bigger.
B) It has doubled.
C) It has decreased by 50%.
D) It has grown by 3,000%.
8) How does the article characterize the drop in UX research job openings from 2022 to 2023?
A) As an inevitable downturn.
B) As a sign of UX dying.
C) As fake data because job openings were stable in 2023.
D) As a misleading cherry-picked number relative to the trend across many years.
9) What is Nielsen's Corollary to Saffo's Law?
A) Don't assume a blurry view means a long distance.
B) UX is a dying profession.
C) The future is uncertain for UX.
D) Never assume salaries will rise indefinitely.
10) How does the article view UX professionals who started their careers in bootcamps compared to those with Ph.D.s?
A) Bootcamp professionals are superior.
B) Both can bring value to the field.
C) Ph.D. professionals are the only ones respected.
D) Bootcamp professionals are unqualified.
11) What is the main argument against the notion that UX is in a state of decline?
A) The field is too new to judge.
B) The article argues that UX is in decline.
C) UX maturity has improved every decade.
D) The article doesn't argue against the notion.
12) What does the article predict about the number of UX professionals worldwide by 2035?
A) 10 million
B) 3 million
C) 1 million
D) 500,000