Where Does Craft Go in the Age of AI?

Where Does Craft Go in the Age of AI?

Starting Craft of Research (2)

The last article was about the craft of research. This one looks for hints about where that craft moves next, in the examples of other industries.

Why the Most Automated Car Plant Still Keeps Its Masters

The Lexus Tahara plant in Japan's Aichi prefecture is counted among the most automated production facilities in the world. And yet the plant keeps a group of people called takumi: master craftsmen who have put sixty thousand hours, eight hours a day for thirty years, into a single skill.[1]

What's interesting is their relationship with the robots. Just before painting, there's a process in which the car body is wet-sanded. The hand movements that produce a perfect surface are something only a seasoned craftsman knows, so the robots in this process are trained to reproduce the takumi's strokes. And on the paint inspection line, the takumi deliberately place a flawed body on the line every day to test the team's eyes. Not once, so far, has a flaw made it through.[1]

The most automated plant, rather than sending its craftsmen away, teaches their sense to the machines and verifies the machines with their eyes. Automation and craft are not opposites.

Technology Raises the Floor of Work

That AI raises the floor of work is now a fact confirmed by data. In a 2023 experiment that researchers from Harvard Business School and Boston Consulting Group ran with 758 consultants, those using GPT-4 finished 12.2 percent more tasks than those without, finished them 25.1 percent faster, and scored significantly higher on quality. The gains were largest where the baseline was lowest: consultants in the bottom half improved by 43 percent, against 17 percent for the top half.[2] AI worked in the direction of closing the gap, which is to say, raising the floor.

But the same experiment holds a less-known result. The researchers slipped in one task designed to sit just outside AI's range of competence, and there the outcome flipped: the group using AI was 19 percentage points less likely to get the answer right.[2] Faced with a task it couldn't do, the AI answered confidently and plausibly anyway, and people failed to filter it out.

The researchers named this the jagged frontier. The line between what AI does well and what it doesn't does not match apparent difficulty, so you can't tell from the output alone whether you're inside the line or out. I've argued that the core ability in craft is not making but recognizing, the eye that tells the good from the merely plausible. What this experiment shows is precisely that the price of that eye is going up. The easier making becomes, the easier the merely plausible becomes, at exactly the same speed.

Craft Doesn't Disappear. It Moves

In the last article I borrowed David Pye's distinction and wrote that every time technology converts a task into the workmanship of certainty, craft moves: automate the planing and craft rises to the judgment of which wood to choose and what form it should take.

That is what's happening in research now. The execution layer, writing questions, running cross-tabs, drafting reports, is fast becoming the workmanship of certainty. Which means craft moves in two directions.

One is upward. Beyond simple execution, it rises to the judgments of what to ask and whether to trust this result. As execution gets cheaper and faster, the odds of executing the wrong question grow with it, so the value of the capacity to define the problem and verify the result will, if anything, climb.

The other is into the tools. Just as the Tahara plant taught the craftsman's strokes to its robots, the expert's standard moves into the tool as its default. Software has its own example of this direction. Harvey, the legal AI company, was built by placing lawyers throughout the product development process, and recently put "Built by Lawyers" front and center.[3] The claim is that the quality of legal AI depends not on the AI technology but on whether the people who know what good legal work is take part in building the product.

The Research Industry Is on That Test Bench Now

In research, I think this question has surfaced most sharply in the debate over synthetic data, or synthetic consumers. The industry has argued over the practice of creating virtual respondents with LLMs and having them answer surveys, and the consensus at this point runs as follows: synthetic data is a complement, valid only when anchored to and verified against real human data, not a substitute. ESOMAR, the industry's international association, revised its code accordingly, putting transparent use of synthetic data and human oversight up front.[4]

The consensus tells you something. What decides the success of AI research is not the AI itself but the craft on the verifying side. Without someone who knows what good data is, there's no way, synthetic or measured, to separate the plausible numbers from the trustworthy ones.

The design industry, wrestling with much the same question, has landed in the same place. Design is often rated the field most threatened by AI, and yet Dylan Field, the CEO of Figma, opened this year's conference keynote by telling the audience that "AI has lowered the floor, but it has not raised the ceiling", and that raising the ceiling was up to them.[5] The industries standing at the center of the replacement discourse are the ones talking about craft the loudest. I think research is standing in exactly the same spot.

The Shift in Craft Comes with Conditions

To pull it together: when AI raises the floor of work, craft doesn't disappear. It moves up into higher judgment and becomes the standard of the new tools. But this shift doesn't happen on its own. Higher judgment comes from accumulated experience. Just as it takes someone who has done the planing to choose the wood, no one can judge what to ask, or whether to trust a result, without having passed through project after project. And the transplant into the tools is possible only when the people who hold the standard and the people who build the tools are one body. The robots at Tahara could learn the craftsman's strokes because the craftsmen were inside the plant.

The same holds for Opensurvey. If the organization that knows research and the organization that builds the technology were separate, keeping up with the age of AI would not have been easy. In the next article, I want to tell the story of Opensurvey, where the two have been one body from the start. It's the story of the team I've watched for the past fifteen years.

References

[1] "Could you become a Takumi? In Japan it takes 60,000 hours to reach the highest level of craftsmanship," Lexus Europe Newsroom, March 2019. On the wet-sanding robots and paint inspection: Discover Lexus, "The Harmony of Man and Machine" and "There Are No Shortcuts."

[2] Fabrizio Dell'Acqua, Edward McFowland III, Ethan Mollick, Hila Lifshitz-Assaf, Katherine Kellogg, Saran Rajendran, Lisa Krayer, François Candelon, and Karim R. Lakhani, "Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality," Harvard Business School Working Paper 24-013, 2023. Later published in Organization Science, 2025.

[3] "Built by Lawyers, Tailored by You," Harvey blog, May 2026. On placing lawyers in the development process, see OpenAI's customer story "Customizing models for legal professionals."

[4] On the ICC/ESOMAR code revision and synthetic data provisions: "AI in Market Research: Five rules to live by," Research World (ESOMAR), August 2025. On industry views of synthetic respondents: Rival Group, "2026 Market Research Trends Report," December 2025.

[5] Dylan Field, opening keynote, Figma Config 2026, June 24, 2026.