What Is Craft?

What Is Craft?

The sociologist Richard Sennett defined craftsmanship as "the desire to do a job well for its own sake," an impulse he called enduring and basically human.[1] Craft, in other words, is not a particular skill but an attitude toward quality: doing your best even when no one is watching, even when the result will never show it. Sennett's whole definition fits into two words. I'll know.

Craft Lives Where There's Risk

The furniture design theorist David Pye drew a line between the workmanship of risk and the workmanship of certainty.[2] Plane a board by hand and every moment rests on the maker's judgment, but stamp it out with a factory press and the machine guarantees the outcome. Craft belongs to the first kind, the work whose result depends, moment by moment, on judgment and skill.

So does technological progress make craft disappear? No. Every time technology converts a task into the workmanship of certainty, craft simply moves. Say the planing gets automated: the craft that used to live in each stroke rises to a higher question, which wood to choose and what form it should take.

The Body of Craft Is Knowledge That Was Never Put Into Words

You can't talk about craft for long without arriving at tacit knowledge, the idea that human experience and understanding never transfer cleanly into language. The philosopher Michael Polanyi introduced the concept with a single line: "We can know more than we can tell."[3]

A craftsman is not someone who follows the manual but someone who knows what the manual can't hold. Ten thousand repetitions don't make the work faster; they build the sense that something is off. So the core ability in craft isn't making. It's recognizing, the eye that tells the good from the merely plausible.

Craft Is Finished by Restraint

At the summit of the crafts sits Joseon white porcelain: nothing but a simple line and a white glaze, and yet what we feel in it is a universal kind of beauty. Great painters, Picasso among them, tend to get simpler with age. What countless attempts finally teach is not what to add but what to take away. The beginner does everything they can. The master knows what not to do.

A well made thing looks easy because it's seamless, but packed inside it are countless deliberations, decisions, and years of experience. Craft is the sum of the judgments you can't see in the result.

Since AI arrived, every industry has been talking about what gets replaced and what remains. The planing is being automated fast. But return to Pye's distinction and you find that after the automation, the person who can make the higher judgment technology couldn't take is, in the end, the person who has done the planing, because the eye that separates the good from the plausible comes from a sense built up in the body. Which is why, paradoxically, now is the best possible moment to talk about craft.

And research, the field I've watched for the past fifteen years, is where this story fits unusually well.

What Is the Craft of Research?

It's in What You Built Where No One Can See It

Nobody sits in a chair that looks rickety. The dangerous chair is the one that looks fine but is missing a screw, and research works the same way: give it data and you'll get response rates, charts, a finished report, none of which will tell you whether the research was any good.

Turn that around and it means the quality of good research also accumulates where you can't see it, in the small judgments that define the problem, choose the words, and build the quality of the interpretation. These judgments come not from theory but from over the shoulder, from a sense earned across project after project. Which is another way of saying that research still holds a great deal of tacit knowledge no one has written down. This is why craft matters in research.

(It's also why craft is hard to talk about. The gap in tacit knowledge is not something you can see.)

The Craft of Framing: Defining the Problem Properly

The statistician John Tukey famously argued that an approximate answer to the right question beats an exact answer to the wrong one.[4] The greatest failure in research is not the wrong answer but the exactly right answer to the wrong question.

Knowing when to ask "is this ad good?" and when to ask "do you remember this ad?" Asking back what real decision is hiding behind the question the requester brought you. This capacity to frame the problem is tacit territory, and it grows only through long experience with real projects.

The Craft of Design: Knowing the Weight of a Single Word

Survey design turns a defined problem into questions, and at the same time it simulates, in advance, what will happen inside the respondent's head. One word changes the result. In a Pew Research Center experiment, sixty percent of people said their area had enough jobs, but when asked whether it had enough good jobs, the figure fell to forty eight.[5] That the same policy draws different answers depending on whether you ask "should this be forbidden" or "should this be allowed" is a classic finding, verified across decades of survey methodology.[6]

The priming created by question order, the normal range a set of options quietly implies, the point where respondent fatigue begins to contaminate the data: whoever designs the survey has to see all of it coming. A good questionnaire is short not out of laziness but because everything that could be cut has been cut.

The Craft of Interpretation: Knowing What the Data Does and Doesn't Say

Not the ability to read numbers but the ability to know their limits: an awareness of what this sample can and can't support, the difference that is statistically significant and practically meaningless, the gap between what a respondent says and what they'll actually do.

Interpretation fails in two directions, the overreach that says more than the data says and the evasion that won't say what the data does say. A good researcher stands between the two, understands what the data is saying, and translates it so the people using it can make a better call. This takes considerably more than writing a report and handing the judgment off to whoever reads it.

The Craft of Responsibility: Not Forgetting the Respondent

On the far side of the data is a person who gave up their valuable time. A good researcher thinks about a survey length that respects that time, the courtesy of asking only what can be answered, and the quality of the response experience itself.

This is an ethic the researcher should hold, and it is also a methodology for getting good data, because a bored and tired respondent is a human being, and human beings leave inaccurate answers. Data gets collected whether or not you consider the respondent, of course. Whether that data becomes an answer you can trust is another matter.

The Answer You Can Trust and the Answer That Merely Sounds Right

Follow all of this through and the craft of research turns out to be something more than the skill of producing a good answer: the ability to tell the answer you can trust from the answer that merely sounds right. That ability comes from an attitude toward quality, from choosing the word even when it won't show in the report and designing for a respondent you'll never meet. The I'll know attitude.

Before the next article, one thing I want to make clear. The craft I'm describing is not a stubborn attachment to old methods. Just as craft moved upstream when the work got automated, what you protect when the tools change is not the method but the standard, and taking on new technology while holding the standard is, I believe, the thing the people who build the tools themselves can do best.

Opensurvey works on moving the experience and instinct the artisan has built up into tools everyone can use. I wrote that craft is the sum of the judgments you can't see in the result. If so, then to talk about craft is to make those invisible judgments visible. In Craft of Research, we record and open up the countless decisions behind making research, and the tools for it.

The next piece takes up this question: where does the craft of research move now? I'll look to other industries for a clue.

References

[1] Richard Sennett, The Craftsman, Yale University Press, 2008, p. 9.

[2] David Pye, The Nature and Art of Workmanship, Cambridge University Press, 1968. The distinction between the workmanship of risk and the workmanship of certainty appears in chapter 1.

[3] Michael Polanyi, The Tacit Dimension, Doubleday, 1966, p. 4.

[4] John W. Tukey, "The Future of Data Analysis," The Annals of Mathematical Statistics 33(1), 1962, pp. 13-14.

[5] Adam Hughes and Bradley Jones, "'Good jobs' vs. 'jobs': Survey experiments can measure the effects of question wording, and more," Pew Research Center, January 29, 2019.

[6] The forbid/allow asymmetry. Original experiment: Donald Rugg, "Experiments in Wording Questions: II," Public Opinion Quarterly 5(1), 1941. Meta-analysis: Bregje Holleman, "Wording Effects in Survey Research: Using Meta-Analysis to Explain the Forbid/Allow Asymmetry," Journal of Quantitative Linguistics 6(1), 1999.