Overview of Methodology Used to Produce The Lexical Tendencies

Here is a brief description of our methodology to produce the results of the lexical tendencies analyses.

Measuring lexical tendencies involved a number of steps, the first of which was to construct customized versions of James Pennebaker’s Linguistic Inquiry and Word Count Libraries. Each library is composed of a specific number of terms and each term is applicable to the measuring of a specific psychological dimension. Our versions range in size from 23 words to just over 900, the disparity in breadth a result of careful inclusion by Pennebaker and his team of expert judges. The terms were each hand selected by Pennebaker and a team of expert judges who exhaustively combed through various, entire dictionaries to determine the applicability of every word. Each term in every library was empirically supported to measure one or more of the 10 psychological characteristics we examined: anger, happiness, sadness, power, affiliation, achievement, conviction, analytical thinking, formal thinking, and narrative thinking.

Once the libraries were built, we then used the Python program to extract the function words from every individual tweet and then compared the numbers of function words to total words used. We generated proportions based on these comparisons, giving us stable readings of each tweeter’s use of each word group. These proportions were created at the individual user level, to be later averaged and used when measuring the three primary factions in our study. Wanting to make sure we had enough real data to perform stable, authentic analyses, we chose to remove any individual who tweeted less than 15 function words (approximately one tweet) over the entire one-year period.

Next, we sought to average across the multiple libraries within psychological characteristics that included more than one library. Since 7 of the 10 psychological characteristics are measured by more than one word library, some of them by 3 and others by up to 13, we standardized the function word proportions using z scores (µ=0; s.d.=1). This served to essentially equalize the differences in size between the different libraries, and having done this, we were able to produce true averages for each individual word group.

Our next step was to connect every individual tweeter to one of the three factions previously identified by our network analysis (see Giant Network). Essentially, our social network analysis program determined the faction to which each individual belonged, based upon their behavioral choices on Twitter. That is, people were connected to groups because of their choices to follow, retweet, or mention others on Twitter. Using these data, we categorized the individual tweeters by three colors: green (supporters of the Common Core), blue (opponents of the Common Core from within education), or yellow (opponents of the Common Core from outside of education).

Using these groups, and the standardized scores from the individuals comprising these groups, we then performed an analysis of variance (ANOVA) to test for significant differences in each psychological characteristic.

For our final step, we decided to report function word use in number of words used per 100 rather than in standardized scores, because we believed this would be more meaningful to our readers. Thus, we chose one of the libraries in each psychological characteristic as an anchor and, using a linear transformation, converted the standardized scores into the metric for the anchor library. This involved adding up the number of words in that one psychological characteristic and reporting the results as the number of words per 1000. A consequence of this approach is that those characteristics with more libraries resulted in a larger number of words per 1000, because there are more words that can be identified within each dimension. Therefore, we caution readers not to compare the frequency of words used across the psychological characteristics, but focus instead on comparing the numbers for each faction within each dimension.

Our methodology is described in even greater detail in the project methodology.