What Is Craft?

What Is Craft?

The sociologist Richard Sennett defined craftsmanship as "an enduring, basic human impulse, the desire to do a job well for its own sake."[1] Doing something well even when no one is watching, even when it won't show in the final result. Craft is not a particular skill but an attitude toward quality.

"I'll know" — those two words could stand in for Sennett's whole definition.

Craft lives where there is risk

The furniture-design theorist David Pye distinguished the "workmanship of risk" from the "workmanship of certainty."[2] When you plane wood by hand, the outcome depends on the maker's judgment at every moment; when you stamp it out with a factory press, the outcome is guaranteed by the machine. Craft belongs to the former — work in which the result depends on judgment and skill exercised moment by moment.

This distinction matters because every time technology turns some task into "workmanship of certainty," the location of craft shifts. When planing is automated, craft doesn't disappear; it moves upstream, into the judgment of "which wood to choose, what form it should take."

The substance of craft is knowledge that hasn't been put into words

The philosopher Michael Polanyi left us the concept of tacit knowledge with the phrase, "We can know more than we can tell."[3] It means that human experience and knowledge can never be fully rendered into language.

An artisan is not someone who follows the manual but someone who knows what can't be written into one. What ten thousand repetitions build is not speed of execution but the sense that "something is off." That is why the core ability of craft is not making but recognizing — the eye that tells the good from the merely plausible.

Craft is completed through restraint

At the summit of craft sits Joseon white porcelain. It is nothing but simple lines and white light, yet we feel a universal beauty in it. The paintings of great artists like Picasso tend to grow simpler with age. What you come to understand at the end of countless attempts is not what to add but what to remove. The beginner does everything they can; the master knows what not to do.

Something well made looks easy because it is seamless. But compressed inside it are countless deliberations, decisions, and experiences. Craft is the sum of judgments that never show in the result.

Since the arrival of AI, every industry has been talking about what will be replaced and what will remain. Hand-planing is being automated fast. But recall Pye's distinction: after automation, the person who can make the upstream judgments is, in the end, someone who has actually done the planing. The eye that tells the good from the plausible comes from a sense built up in the body. So, paradoxically, I think now is the best possible time to talk about craft.

And the research industry I've watched for the past fifteen years is a field where this story fits especially well.

What Is the Craft of Research?

It lies in what has been built up, even when unseen

No one sits in a chair that looks flimsy. The dangerous one is the chair that looks fine but is missing a screw. Research is the same. If you have data, you get response rates, you draw charts, you finish reports. From the output alone, it's hard to tell good research from bad.

Put the other way: the quality of good research is likewise stacked up where it can't be seen. Defining the problem, choosing the words, drawing the lines of interpretation — the large and small judgments. These come not from theory but from a sense picked up over someone's shoulder, learned project after project. Which is to say, research still holds a great deal of undocumented tacit knowledge.

This is why craft matters in research — and why craft is hard to talk about.

The craft of problem definition — defining the problem properly

The statistician John Tukey has a famous line: "Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise."[4] Research's greatest failure is not a wrong answer but a precise answer to the wrong question.

Distinguishing when to ask "Is this ad good?" from when to ask "Do you remember this ad?" Asking back what the real decision is, hidden behind the question the person who requested the data brought you. This capacity for problem definition is precisely the domain of tacit knowledge, and it is cultivated only through long project experience.

The craft of design — knowing the weight of a single word

Survey design is the work of turning a defined problem into questions, and at the same time of simulating in advance what will happen inside the respondent's head.

A single word changes the results. In a Pew Research Center experiment, when people were asked whether there were enough "jobs" in their area, 60% said yes — but when asked whether there were enough "good jobs," it dropped to 48%.[5] That the same policy draws different responses when asked "Should it be forbidden?" versus "Should it be allowed?" is a classic phenomenon, verified over decades of survey methodology.[6]

The priming created by question order, the normal range a set of options quietly implies, the point at which respondent fatigue begins to contaminate the data — the person designing a survey has to see all of this in advance. A good questionnaire is short not out of carelessness, but because everything that could be removed has been removed.

The craft of interpretation — knowing what the data does and doesn't say

Not the ability to read numbers, but the ability to know their limits. Awareness of what this sample can and cannot speak to; differences that are statistically significant but practically meaningless; the gap between what respondents say and what they will actually do.

Failures of interpretation come in two kinds: the overstatement that says more than the data says, and the evasion that fails to say what the data does say. A good researcher, between the two, understands what the data says and interprets it so the user of that data can make a better decision. This demands far more skill than simply writing a report and handing the judgment off to whoever reads it.

The craft of responsibility — not forgetting the respondent

On the other side of the data is a person who gave their precious time. A good researcher considers a survey length that respects the respondent's time, the courtesy of asking only what can be answered, and the quality of the response experience itself.

This is both an ethic and a methodology for obtaining good data. A bored, tired respondent — being human — may leave inaccurate answers. Of course, data gets collected even without any care for the respondent. Whether that data becomes an answer you can trust is another matter.

Trustworthy answers and plausible answers

If I tie together everything so far, the craft of research seems to be less the skill of producing a good answer than the ability to tell a trustworthy answer from a plausible one. And that ability comes from an attitude toward quality — the "I'll know" attitude of choosing your words even when it won't show in the report, of designing for the respondent even when you'll never meet them.

Before moving on to the next article, there's something I want to make clear. The craft I'm describing is not about clinging to old methods. Just as craft moved upstream when technology automated a task, what must be preserved even as tools change is not the method but the standard. To adopt new technology while holding to the standard — I believe that is something the people who build the tools themselves can do best.

Opensurvey is doing the work of translating the experience and sense that artisans have built up into tools everyone can use. I wrote that craft is the sum of judgments that don't show in the result. If so, to talk about craft is to make those invisible judgments visible. In Craft of Research, we record and share the countless decisions in the process of making research — and the tools for it.

In the next piece, we'll take up this question: Where is the craft of research moving now? We'll look for clues in other industries.

References

[1] Richard Sennett, The Craftsman, Yale University Press, 2008. Original: "Craftsmanship names an enduring, basic human impulse, the desire to do a job well for its own sake." (p. 9)
[2] David Pye, The Nature and Art of Workmanship, Cambridge University Press, 1968. The distinction between the "workmanship of risk" and the "workmanship of certainty" is presented in Chapter 1.
[3] Michael Polanyi, The Tacit Dimension, Doubleday, 1966. Original: "We can know more than we can tell." (p. 4)
[4] John W. Tukey, "The Future of Data Analysis," The Annals of Mathematical Statistics, 33(1), 1962. Original: "Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise." (pp. 13–14)
[5] Adam Hughes & Bradley Jones, "'Good jobs' vs. 'jobs': Survey experiments can measure the effects of question wording – and more," Pew Research Center, January 29, 2019. https://www.pewresearch.org/short-reads/2019/01/29/good-jobs-vs-jobs-survey-experiments-can-measure-the-effects-of-question-wording-and-more/ (jobs 60% vs. good jobs 48%)
[6] The so-called forbid/allow asymmetry. First experiment: Donald Rugg, "Experiments in Wording Questions: II," Public Opinion Quarterly, 5(1), 1941. Meta-analysis: Bregje Holleman, "Wording Effects in Survey Research: Using Meta-Analysis to Explain the Forbid/Allow Asymmetry," Journal of Quantitative Linguistics, 6(1), 1999.