collecting tokens

The word token has many meanings, having synonyms such as symbol, memento, or representative:

    1. I give you this squid as a token of my affection.
    2. I’ll keep these pants forever as a token of my holiday escapades.
    3. I posted this photo of a duck in the dishwasher as a token of the many pictures I’ve taken of random things.

A token can also be a conventionalized object, such as a metal coin or plastic figure, used in place of money for some transactions or used in some sort of group activity, like a game.

    4. I’m not sure what to do with my old subway tokens now that they’ve started using Charlie Cards.
    5. My old Monopoly game was missing half its tokens.

In my world, though, the most frequent use of the word token is the meaning used in linguistics. (Interestingly, the page, with all its various links and definitions from those various sources, doesn’t even mention linguistics.) In linguistics, a token is an instance of some form that is being studied, an item of a particular category or class. It is commonly discussed in terms of the type-token distinction, which has its roots in philosophical usage:

Type (metaphysics)

A type is a category of being. A human is a type of thing; a cloud is a type of thing (entity); and so on. A particular instance of a type is called a token of that thing; so Socrates was a token of a human being, but is not any longer since he is dead. Likewise, the capital A in this sentence is a token of the first letter of the Latin alphabet.

According to the Stanford Encyclopedia of Philosophy,

The distinction between a type and its tokens is an ontological one between a general sort of thing and its particular concrete instances (to put it in an intuitive and preliminary way).

In linguistics (and in related speech and language research) the term token is used to refer to any single instance of some phenomenon or category that’s under investigation, and type is used for some category of which a token is a member. The type-token distinction is often used when investigating words used in a written text. Imagine, if you will, a short text such as:

I like the word pants. I actually like saying the word pants. It’s one of those words that begs to be repeated. Pants. For example, in a discourse on pants, I would hypothesize that speakers would be less inclined to use pronouns to refer to pants than, say, other entities in the discourse. Even if the word pants had just been mentioned, I would still say “pants.”

The text in the block quote above has 63 words. However, it doesn’t have 63 unique words. It has fewer unique words, or word types. I counted 41 unique words, so 41 types. (Mind you, I’m counting things like “say” and “saying” as different words for these purposes, and ignoring punctuation and capitalization.) If we want to look at a particular word type, oh, let’s say maybe the word pants, we can count 7 instances of that word in the text. That’s 7 tokens of pants.

While token is commonly used for a written instance of a word in a text, it can also be used for a larger or smaller unit of speech or language. It could be a spoken production of a sentence, or a production of a single sound segment, like a consonant or a vowel. It could be a gesture. It all depends on what categories, or types, that you are looking at.

For example, let’s say I’m studying phonetic characteristics of a vowel in American English, such as [æ], the vowel in words like bad, pat and pants. I would probably want to collect a large number of instances of words spoken aloud that contain that vowel. If I get a recording of someone reading a list of 5 words with [æ], and I have them read that list 3 times, I end up with 15 tokens of [æ] by that speaker. I could also talk about having 15 tokens of words containing [æ], or even 15 tokens of utterances containing [æ]. If I have 4 speakers all reading that same list, 3 times each, I end up with 60 tokens of [æ].

Here’s an example of the use of the word tokens from a phonetics paper* I grabbed off the web (found by googling “tokens of p”, in case you’re wondering):

This includes all /k/ and /p/ tokens produced, not only those in potentially fricatable environments.

(And yes, I do get off on this stuff.)

The article repeatedly mentions tokens of /p/ and tokens of /k/, and how many tokens of each fit some criteria, or follow some pattern.

Now let’s say we wanted to study the use of the word tokens in that text. (So in this case, our type is tokens.) Using a basic text search, I counted 28 instances of the word tokens. That means that the text contains 28 tokens of tokens.

Much of what I do as part of my research, especially for my various jobs, involves collecting, categorizing and otherwise analyzing tokens. I love this part, collecting and working with the data. It’s the thrill of the hunt. Followed by the thrill of the puzzle. Followed by the thrill of the data organization. (What I must learn to love is the thrill of the write…)


*Loakes, D. and McDougall, K. (2004) “Frication of /k/ and /p/ in Australian English: Inter – and Intra-Speaker Variation” in Proceedings of the 10th Australian International Conference on Speech Science & Technology, pp 171-176.

Nimberpoop, R. (1954) “What’s your deal with the word pants? A study in bizarre philological obsessions.” Sense, Nonsense and Polysemy Quarterly, 3, pp. 4-97.

5 thoughts on “collecting tokens

  1. Jen, she’s actually a robot. And she can shoot laser beams from her eyes, too, but she hardly ever does that since the incident with the squid.

  2. Jaŋari-
    Thanks. I have fun with the banner. It started off just being due to trouble committing to a banner image, and then developed into a plan. Plus it lets me play around with photos I otherwise would never likely use.

    You’re funny.

    I thought I’d erased that from your memory…

  3. I think it clarify more my doubts about tokens. I’m studying linguistics, and some things are so confusing. I have to do a homework related to word tokens for the types GIVE and FORGET. I have only found 2 tokens for the sound /g/

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s