Laura P. Wolton*, University of Colorado Denver, School of Public Affairs
Deserai A. Crow, University of Colorado Denver, School of Public Affairs
Tanya Heikkila, University of Colorado Denver, School of Public Affairs

To cite the chapter: Wolton, Laura P., Deserai A. Crow, and Tanya Heikkila. 2022. “Stepping Forward: Toward a More Systematic NPF with Automation”, in Narratives and the Policy Process: Applications of the Narrative Policy Framework, Michael D. Jones, Mark K. McBeth, Elizabeth A. Shanahan (eds.), Montana State University Library, 40-90. doi.org/10.15788/npf3

Introduction

      For the last 60 years, text analysis methods have allowed advances in the research traditions of various fields, including behavioral sciences, consumer research, and media analysis (Humphreys &Wang, 2017). The General Inquirer (Stone et al., 1962) was one of the first computer-assisted approaches designed for the analysis of text, employing a dictionary lookup method that “tagged” a sentence for the appearance of a word belonging to a specific category, and became a precursor to dictionary lookup methods available to content analysts today. Recent computer-assisted text analysis advances include semi- to fully-automated approaches, which emerge from multi and interdisciplinary research efforts from the computer sciences, social sciences, linguistics, and artificial intelligence.

     While automated and semi-automated text analysis methods advanced for decades, we have only recently seen policy process scholars using these methods in published research (e.g., Lawlor, 2015; Heikkila &Weible, 2017; Olofsson et al., 2018; Berardo et al., 2020; Blair et al.,2016; Scott et al.,2020). Although some of these applications of automated text analyses have looked at themes and issues that arise in policy debates, less attention has focused on the structure of policy narratives. With widespread availability of policy narrative content in various media formats—from online news, downloadable publications, to social media—there is an ongoing need for automated tools to systematically evaluate larger datasets of narratives (Shanahan et al., 2018). This chapter will demonstrate that the incorporation of semi-automated and automated methods into the policy researcher’s analytical toolbox can advance policy narrative research. The chapter begins by discussing the differences and connections between narratives and frames, beginning with a detailed discussion of policy narratives. The example case used in this analysis is then described, followed by a step-by-step description of the automated method introduced in this chapter. The findings are presented followed by a discussion of the implications of this method for policy process scholarship in general and the study of narratives specifically.

     The employment of automated methods in policy narrative research is congruent with the ambitions of the Narrative Policy Framework (NPF) as the framework itself promotes a systematic structure that allows for increased generalization and comparison amongst research findings. Through its operationalization of narrative elements, the NPF allows for a systematic and reliable investigation into the role of narratives in the policy process. As of yet, only two studies have used semi-automated techniques with the NPF—both with inductive methods of categorization for designing autocoding dictionaries (Merry, 2018; Crow & Wolton, 2020). The first objective of this chapter is to extend the previous work done by Crow and Wolton (2020) and, in doing so, propose a method to connect autocoded policy actors to NPF characters.

     The second objective of this chapter is to investigate the relationship between frames and narrative elements, expanding on an emerging body of NPF literature (Jones & Song, 2014; Lawlor & Crow, 2018; Crow & Lawlor, 2016; Merry, 2018). The framing literature suggests that communications using the same frame will contain the same narrative elements. For example, within one frame, similar categories of policy actors are likely to be portrayed in a similar character role (e.g., industry actors as villains, government actors as heroes). The solutions within one frame should similarly remain constant. Characters and solutions are likely transported in frames together because actors have a range of solutions that they have the ability or authority to perform (e.g., government can regulate, citizens can vote). Thus, to our second objective we ask: How does a narrative frame relate to the characters used and solutions presented? In answering this question, we not only offer insights on semi-automated narrative analysis tools, but also offer theoretical advancements for the NPF and framing literature.

NPF, Framing, and Autocoding

     Arguably, improving automated techniques will amplify the potential for systematic and generalizable policy narrative research, particularly if the methods are transparent and accessible. Frameworks create a shared orientation and conceptual map that structure how researchers study a particular phenomenon (McGinnis & Ostrom, 2014). For policy communication researchers, the development of the NPF (Jones & McBeth, 2010) was a successful step towards promoting an increased degree of structure and consistency in policy narrative analysis. The interdisciplinary nature of policy narrative research and, thus, the potential for application of theories across policy domains highlights the need for increased replicability and validity in NPF studies (Shanahan et al., 2018).

    Though NPF analysis has not commonly included frame analysis, several scholars have made efforts to connect the framework with framing theory (Jones, 2013; Crow & Lawlor, 2016; Olofsson et al., 2018). Frames are broader approaches to narrating a topic, focusing attention on particular themes, ideas, or issues. In doing so they can constrain narratives to the selection of a particular set of narrative elements that are commonly aligned with a given frame. Recent advances in the automated detection of frames in news articles suggest that, coupled with an increased understanding of elements, these methods can expand our understandings of the structures and elements of policy narratives.

Narrative Policy Framework

    The NPF is a means of investigating the structure and content of policy narratives (and the implications of that content to public opinion and policy) both qualitatively and quantitatively, promoting shared units of analysis and codebook design. The NPF asserts that policy narratives are structured stories that include a setting, characters, plot, and a moral of the story (the policy solution to the problem). However, to be considered a policy narrative, a communication or text must contain at least one character and a reference to a policy issue (Jones et al., 2014).

Characters

    The term characters, according to the NPF definition, involves those entities “who act or are acted upon” and are categorized according to common roles that appear in narratives (Shanahan et al., 2018, p. 335).  While only three narrative characters appear in the original framework of the NPF (heroes, villains, and victims), subsequent studies have added to the range of identifiable characters (Shanahan et al., 2017). Of those previous categories, this study limits character identifications to heroes, villains, victims, allies, opponents, and charismatic experts:

Moral of the Story or Solution

     One of the four primary narrative structure elements (i.e., variables) in the NPF is the moral of the story, which elicits action or provides a policy solution (Stone, 2002; Ney & Thompson, 2000; Verweij et al., 2006). Solutions are most often related to characters in the NPF, as heroes “take action with purpose to achieve or oppose a policy solution” (Shanahan et al., 2018, p. 343). NPF research shows that though not necessary to the definition of policy narrative, solutions are present in a high proportion of narratives (Crow et al., 2017; Crow & Wolton, 2020). Research indicates that solutions are more likely to be accepted by readers who experience positive affect towards story characters (Jones & Song, 2014), pointing to the need for further research on both characters and solution. Thus, in exploring how frames relate to characters and solutions, we can help advance this line of research.

Frames

     Framing theory from the mass communication scholarship helps us connect the broad issue characteristics of focus (i.e., frames) to the more specific structural elements (e.g., characters, solutions) of policy narratives. As Crow and Lawlor (2016) discuss, frames are the overarching approach to narrating a story that constrains the specific choices made in construction of narratives. For example, when framing a discussion of the climate change issue, a narrator could use a religious frame and, therefore, cast characters such as a pastor or God when discussing the morality of addressing the climate crisis. While both policy narratives and frames may be used strategically to influence policy outcomes, narratives more overtly include story-like elements such as morals and contrasting characters.

    Framing entails the “selection of certain aspects of a perceived reality and makes them more salient in a communicating text, in such a way as to promote a particular problem definition, causal interpretation, moral evaluation, and/or treatment recommendation” (Entman, 1993, p. 52). That is, there is a larger structure to any story that organizes facts into a storyline and emphasizes selected facets to communicate a particular viewpoint (Crow & Lawlor, 2016). Framing theory considers frames to be the structural “bones” of stories—a major theme that bounds the story within which the narrative elements are assembled.

     An underlying assumption of framing theory is that there are options for how to present information (Scheufele & Iyengar, 2014). Thus, the effects of framing on audience perceptions are due to choices made in the way information is presented—not necessarily the facts that are revealed. Journalists are not the only communicators who frame stories—individuals do the same in how we convey information and focus on selective parts of a story at the expense of other potential angles. For example, journalists in the United States strive for “unbiased” presentation of facts but are also taught to shape stories in an audience-focused manner; to stir emotion and empathy they select from a list of issue characteristics choosing the most compelling (Crow & Lawlor, 2016). These choices are consequential because focusing on one aspect of a policy issue may limit the conversation of the public and policymakers around that issue (Crow & Lawlor, 2016; Lawlor, 2015).

     There are a number of approaches to studying frames and competing or conflicting definitions. Using framing analysis, we can study sub-topics of a broad issue (issue frames), different conceptual aspects of an issue from moral to political and others (news frames), broad themes versus specific incidences (thematic vs. episodic frames), among others. In this analysis, we will employ issue frames, but others could be used depending on the focus of a given study. A researcher needs to specify the approach and follow the method presented in this chapter with a clear definition of frames in mind.

Frames and Elements Theoretically Travel Together

     The definition of framing presented by Entman (1993) suggests that elements of the NPF, such as solutions and characters, should be correlated to the use of frames because framing involves choosing (perhaps not always deliberately) a certain way to present information (e.g., religious vs. economic). The co-occurrence of narrative characters and solutions within a specific frame is a somewhat obvious deduction. For example, if the Environmental Protection Agency is cast as the hero character in an environmental frame, the solution attached to this hero will likely be one related to regulation—the agency has only a range of policy solution activities that it can perform. If the agency is cast as a villain in an economic frame, it may be associated with a similar solution—regulating—while the hero of the story is most likely opposing this government intervention. Finding empirical evidence that narrative elements and frames co-occur would be a meaningful contribution to framing literature as well as NPF so that we can better understand the relationship between these two communication tools and eventually the corollary effects of those tools in tandem and separately.

Applying Automation to Policy Narratives

     While automation has been increasingly applied to other areas of research, automated analysis with the NPF has been held back by some aspects of the framework itself, as well as limitations of automated text analysis in general. We address a few perceived roadblocks and propose ways to move through them, focusing on the NPF elements of characters and solutions. Although the automated detection of frames is relatively established, we also stress the importance of conceptual clarity prior to semi-automated or automated coding.

Character/Actor Association Fluidity

     The NPF defines characters in such a way to allow for fluidity, so that various policy actors or groups may play the role of a single character and that a single policy actor may hold different character roles in different narratives. Because this definitional approach to characters means that one character is never necessarily tied to one policy actor or group, NPF researchers have found it difficult to apply autocoding and capture this fluidity (Shanahan et al., 2018). Our methods, which take the context surrounding the policy actor into consideration, address this issue by allowing the actor to be associated with an NPF character in each frame. Additionally, particularly in cases of large volumes of narratives, the number of policy actors and groups is potentially very large. We use categorization strategies to reduce the complexity of the association between policy actors and characters. Our method takes advantage of information beyond the name of a policy actor to identify and characterize them (further explained in the General Method Description section of this chapter).

Solution Variability

      While the NPF is definitionally clear on the moral of the story—a proposed specific or general policy fix (e.g., “go vote”)—prescriptive coding schema for moral of the story is less developed. Currently, a basic codebook on solutions may include a range of policy solutions that may vary according to policy domain, level of government, or other factors (see Appendix A in Shanahan et al., 2018). For automated coding, this presents a problem with the potentially high variability of solutions, particularly for large volumes of policy narratives. Similar to the work of Crow and Wolton (2020), which created code categories by policy domain, we designed a category schema based on the primary policy tools the solutions are proposed to (or do) implement. Additionally, we allow for solutions that are less concrete than specific actions or policies to be placed in a general category. Our method of autocoding uses parts-of-speech analysis and term-frequency generation as a primary method of identifying solutions (further explained in the General Method Description section of this chapter).

Finding Conceptual Clarity on Frames

      Frames have been effectively identified with semi-automated methods by numerous scholars in various policy domains including public health, immigration, and air pollution (Lawlor, 2015; Olofsson et al., 2018; Poirier et al., 2020; Yu et al., 2020; Berardo et al., 2020). Two general approaches are used in autocoding frames: 1) inductive dictionary development and 2) Latent Dirichlet Allocation (LDA) modeling (these are further explained in the General Method Description section in this chapter). As with any research involving frames, it is essential to be clear which type of frame is under investigation prior to semi-automated or automated coding. Understanding the differences in framing definitions since there are numerous definitions that often conflict or overlap with one another, as well as taking steps to appropriately bound investigation by the objectives of the research, are especially beneficial when approaching large datasets.

The Policy Domain: Shale Oil and Gas Development in the United States

     Energy development, particularly of shale oil and gas, is a contentious issue in the United States. The United States is the world’s top producer of petroleum and natural gas hydrocarbons and has held this position since 2009 when hydraulic fracturing and horizontal drilling allowed for rapid expansion of shale development (US Energy Information Administration, 2018). Proponents of oil and gas development argue that expansion of shale gas provides energy independence, jobs, and economic benefits. For all its beneficial economic contributions, those who oppose shale oil and gas development argue that it can lead to property rights conflicts, environmental harm, and risks to public health and worker safety, among other issues. For instance, some of the recorded impacts of hydraulic fracturing include increased local air pollution from dust, odor, ozone, and volatile organic carbon (VOC) emissions; methane emissions; water contamination; high water use; structural damage from earthquakes triggered by wastewater reinjection; damage to wildlife habitat; abandoned orphan wells; oil, gas, and wastewater spills; noise pollution; and worker fatalities (Joyce & Wirfs-Brock, 2015; Konkel, 2016; Hand, 2015; Witze, 2015; Adgate et al., 2014; Gallegos et al., 2015; Bamberger & Oswald 2015; Moskowitz, 2015; Mason et al., 2015).

     Analyzing news articles on a salient, controversial issue, such as shale oil and gas, is a useful way to explore a wide range of policy narrative elements, such as character portrayals, solutions, and variable frames. Additionally, shale oil and gas development draws significant attention at local as well as state levels in the United States —yielding potentially different narratives across locations and additional variation in the narrative elements used. Lastly, the variance in regulatory stringency of shale oil and gas development across states allows for further insight into a range of policy perspectives.

General Methods Description

     To highlight the potential for semi- or fully-automated techniques and investigate the relationship between NPF elements and frames, we detail a multi-step method. In addition to describing how to detect characters, solutions, and frames, we briefly describe how to check a large data sample of written media to ensure that they are policy narratives. The method presented here is less a procedure than it is a flexible guideline. The researcher’s theoretical lens should ultimately shape the selection of the methodological approach, variables, and relationships to be studied.

     We describe six general phases shown in Figure 1: 1) data selection, collection, and cleaning data; 2) designing and refining the dictionaries; 3) subsetting to only policy narratives; 4) final autocoding; 5) subsetting to a single frame; and 6) mapping of the policy actors into NPF character sentiment ranges. Novice- to intermediate-level programmers will likely do well to complete Phases 1 through 4 in MAXQDA[1], export all segments as a spreadsheet to R, and complete Phases 5 and 6 in R or similar programming language.

Phase 1: Data Selection, Collection, and Cleaning

     Textual data selected for automated content analysis must be in computer-readable formats, the dataset filtered to include only relevant material, and each item cleaned to contain only the text to be analyzed. For text documents, common computer-readable online formats that are also readable by humans include hNews, HTML, XML, and PDF. Some media archive formats, image formats, and hand-written documents are not computer readable and, therefore, will not work with this method of narrative analysis. The process of filtering media, such as removing items that do not fit a study’s selection criteria, can be completed by hand or automatically (as in this study). Options exist for the collection of news articles through news archive services such as NexisUni, application programming interfaces (APIs), or webscraping (automatically downloading webpages). These collection methods may yield relatively consistent retrieved files that can be batch cleaned with a programming language such as R or Python. Cleaning may include the removal of items that will not be analyzed textually, such as emojis, captions, byline material, and other non-relevant content. During the process of cleaning, a catalog file of metadata should be created that contains file names, article titles, source name, date written, author, or other information that is associated with each file.

Phase 2: Design and Refinement of Dictionaries

      In this method, dictionaries are used for autocoding content and take the place of codebooks that researchers use in hand-coding activities. Dictionaries are used to identify and categorize text that is indicative of a theme or concept. Before beginning the discussion of the dictionary development process used in this study, we define several relevant terms and a few technical aspects of designing dictionaries for computer-assisted analysis. Based on Krippendorff (2004) and similar seminal works on content analysis methods, a quantitative content analysis schema contains multiple components: 1) a text corpus—the texts being analyzed, 2) a dictionary—a list of terms associated with theoretical concepts of interest, 3) an unlimited number of categories—groupings of the terms that have shared theoretical characteristics, and 4) rules of identification for each term or phrase such as those for capitalization, plurality, and/or verb forms.

Figure 1. General process outline

     A common way to create dictionaries is in table format, such as the spreadsheet-based dictionary sample in Table 1 that may be read into autocoding software, such as MAXQDA. The sample, taken from the character dictionary for this hydraulic fracturing study, contains a column for the search item, its associated category, and rules for its identification. The rules of identification—whole word, case sensitivity, and starting letters—are entered as on/off (1/0) switches that dictate which forms of a term will be tagged. For example, the search item in row 1, “anti-fracking group,” has been placed in the category activist group. Because “anti-fracking group” has the whole word and case sensitivity toggles turned off, the phrases “anti-fracking groups” as well as “Colorado Anti-Fracking Group” would be tagged during autocoding. The toggle for starting letters, which is irrelevant for the search items in this dictionary sample, becomes meaningful for words with prefixes or that are common portions of words. For example, the toggle would need to be activated when identifying terms like “sent” and “tree” to prevent words like “absent” and “street” from being tagged.

           Considerations about word forms are also essential in dictionary design. Both contextual checks of search items as well as conceptual reviews are necessary to avoid coding errors and design problems. Contextual checks are part of the process of inductive dictionary design (described below) and are beneficial to confirming phrase use, particularly in the case of phrases that potentially have multiple meanings. Reviews of the dictionary for conceptual adherence, as well as consistency in the level of detail of the search items, should occur once at minimum between dictionary development phases.

 ^{a A good resource for working with MAXQDA is Kuckartz, Udo, and Stefan Radiker. 2019. Analyzing Qualitative Data with MAXQDA: Text, Audio, and Video. Switzerland: Springer Nature}

^{b Numerous blogs and resources exist for using text analysis programs in R. A good place to start is this free e-book: Silge, Julia, and David Robinson. 2020. Text Mining with R: A Tidy Approach. O’Reilly Media. https://www.tidytextmining.com/index.html}

One helpful method for generating lists of names and phrases that fit a general pattern is the use of regular expressions or regex, available to those programming in R. For instance, a list of all four-letter acronyms may be found using an expression such as “[A-Z]{4}” in a command like str_extract_all:

str_extract_all(“[A-Z]{4}”,dataframe$text, simplify= FALSE)

Because regex also allows coders to skip words in phrases, grab words of unknown length, or include or disregard punctuation, it can be quite helpful for inductive supplementation as well as tweaking dictionary phrases to capture only the desired forms.

     As a supplemental method for frame dictionary refinement, R coders may use LDA to inductively investigate topical associations using the R package, topic models. LDA provides scores that show how closely words are positioned in the text. The primary reason that we recommend utilizing LDA for frame dictionary supplementation is because it automatically detects the patterns and co-occurrences of words. The resulting topic groupings can be interpreted as ways of framing an issue, as LDA identifies and groups specific keywords used as framing devices (Jacobi et al., 2015). The granularity, or level of detail determined by the number of topics, may result in broader “themes” or finer “issues,” so this number should be conceptually driven. There are several general steps to the LDA process:

First Autocoding

The next step is a test-run of the dictionaries on the text corpus. MAXDictio is specifically designed for the application of dictionaries to a dataset of text, though R programmers may adjust the author’s software in Appendix F for their own use. The tagging of the text will result in the output of a spreadsheet (MAXDictio) or a dataframe with columns containing text segments, located terms, and/or their dictionary categories (Figure 2).

a)

b)

Phase 3: Subsetting to Only Policy Narratives

     NPF analysis is limited to application on policy narratives. As such, a dataset may need to be reduced to only textual data that includes a policy referent. This may be done manually during the data collection. To this end, the researcher may develop a policy referent dictionary to assist in the subsetting process. As modeled in Figure 1, this process includes a priori dictionary development and inductive supplementation followed by the first autocoding run (discussed in Phase 2 above). After the first coding run is complete, the researcher may subset the data to exclude those texts not containing policy referents. At this point, an additional step may be included that involves checking the excluded textual data for policy referents by running a word/term-frequency analysis on only the excluded texts. The resulting list may be searched for additional policy referent terms, which may then be added to the dictionary. If this optional step is taken, the data should be autocoded and subset again, potentially yielding a slightly larger dataset. However, if there are no policy referents in the word frequency list, the step of autocoding again may be skipped.

Phase 4: Dictionary Refinement and Final Autocoding

     Often manual qualitative coding relies on intercoder reliability to provide a measure of internal validity after a codebook has been refined and self-tested by the researchers. However, in an autocoding process using dictionary-based methods, the validity of the results is reliant on the validity of the dictionary. Dictionaries should be checked during construction for context and conceptual coherence, particularly if they are used to assign meaning. We briefly discuss two different dictionary types—counting and interpretive dictionaries—and approaches to increasing internal validity before the final autocoding of the data.

Counting Dictionaries

     Our solutions and policy actor dictionaries are used for tagging nouns, verbs, and proper nouns to identify who, what, where, and counting those occurrences. They are designed to investigate the co-occurrence of elements rather than the symbolism, rhetoric, or messaging embedded in communications. To a much higher degree in the counting than in interpretive dictionary design, the terms are found inductively, and the categories assigned as a result of known information (e.g., policy actor job, solution policy tools). To ensure internal validity for this type of counting dictionary, the terms must fit appropriately into the categories, the terms within one category need to be of similar scale (e.g., all corporations and not individuals and corporations), and cutoff term frequencies must be consistent if every term will not be entered into the dictionary (i.e., terms must occur more than X number of times to appear in the dictionary). This last item, term-frequency cutoff, becomes important as the size of the dataset increases.

     In automated content analysis, although every term may not be captured, the point of using a large size dataset is to capture more generalization (possibly at the expense of nuance). For example, the cutoff for a proper name to be added to the policy actor dictionary must be above 25 times in this study. This helps the researcher to balance time input into dictionary design as well as reduce the unintentional weighting of one category, such as might occur if adding every named individual. Additionally, in the NPF, the names of policy actors in the narratives must mean something to readers to be valuable to a characterization. However, due to the journalistic norm of adding an actor’s role in the same sentence as their proper name (i.e., “Bob Jones, Lafayette resident…”), those actors are not missed in the counting. Thus, the less familiar actors should still be captured by terms for their general policy actor category (e.g., “resident,” “scientist,” “advocate”).

Interpretive Dictionaries

     With dictionaries that provide assignments of meaning to text, such as the NPF and frame dictionaries in this study, internal validity is important, and high intercoder reliability (>95%) is necessary. Assessing the internal validity of an interpretive dictionary includes meeting criteria of both appropriate meaning and mutual exclusivity: 1) Is the term meaningful to the category (i.e., does “criminal” always imply the presence of a villain for all coders?)? 2) Does the term only belong to one category (i.e., does “enemy” fall into both opponent and villain categories)?  With these criteria in mind, the most effective way of analyzing the level of agreement is to provide dictionary terms in random order removed from their assigned categories. Provide the categories available with their definitions as well as the category none to allow coders to reject the term or indicate its overlap into several categories. Researchers may be tempted towards designing larger dictionaries. However, small but meaningful and mutually exclusive interpretive dictionaries are valuable with the large datasets that are typically chosen for automated narrative analysis research—enough instances of a category should be found to be able to create reasonable generalizations.⁴

           Another check for internal validity is to run a correlation and clustering analysis after autocoding with the dictionaries. This check is designed to catch potential overlap of categories, which is especially useful in the case of defining frames. Multiple frames may occur in a single story, but clustering analysis is used to analyze frames at the article level. For NPF characters, articles typically include several character categories and thus clustering analysis may be at smaller unit (e.g., journalistic paragraph or Tweet). Clustering analysis requires the number of counted values for each category per unit of analysis provided—typically as a crosstab, matrix, or dataframe. An example of the results of such an analysis is provided from our study in Figure 4 in the Hydraulic Fracturing Study Methodology section below. Correlation values may be interpreted with the generally accepted ranges of the correlation coefficient in Appendix C.

Final Autocoding

    After the data have been subset and autocoded with the dictionaries, the coded segments are inspected for conceptual coherence as well as validity. Although conceptual coherence should be relatively clear from inspection of the dictionary categories in the dictionary design phase, terms may be catching segments that reveal unexpected results. This may result from a few design issues including 1) incorrect dictionary settings for word or phrase specificity (e.g., only code the whole word, use case sensitivity, or use only starting letters); 2) specifying the incorrect form of a word (e.g., plural, verb endings); or 3) incorrect regex pattern matching. Time spent on refining dictionary terms and settings will result in higher internal validity in the second, and potentially final, round of autocoding.

Phase 5: Subset to Single Frame

     In order to map identified policy actors (or groups of actors) onto NPF characters, we next restrict the dataset to a single frame. The reasoning for this lies in the potential for actors to change NPF character categories in various frames. The process of subsetting may significantly lower the size of the dataset, so the researcher may briefly investigate the descriptive properties of each category to be investigated. Primarily, is the number of data points per character category still adequate to be able to apply rules of normalcy and create reasonable generalizations?5 Not every NPF character category may be fully represented in every frame as some frames are less contentious than others. However, for this single frame, the researcher may choose to use just the characters that are adequately represented.

Phase 6: Mapping of Policy Actors into NPF Character Sentiment Ranges

    With the expectation that the construction of policy actors and characters will differ depending on the frame, we propose a method of relating policy actors to their associated NPF character roles within each frame. The sentiment analysis of sentences containing characters leads to a large volume of scores for each actor and character, which can be related to each other using descriptive and inferential statistical tools. There are several steps to this method: 1) evaluate sentiment in segments tagged as including NPF characters and policy actors, 2) use descriptive statistics to investigate the distribution of the sentiment scores, and 3) relate measures of central tendency and hypothesis testing in order to “map” policy actors onto NPF characters.⁶

1. Evaluate sentiment in sentences within a frame for all NPF characters and policy actors

      In the previous autocoding process, text segments have been tagged with character/actor categories and subset into a single frame. Sentiment analysis is most meaningful at the sentence level but can be evaluated at a slightly larger level of analysis such as the journalistic paragraph, Tweet, or caption. Beyond the paragraph level, sentiment analysis is not a useful exercise unless the analyst is just looking for understanding tone as positive or negative.

      Sentiment analysis is used to evaluate the context around NPF characters and policy actors in this study. Sentiment analysis refers to the identification of emotions, positive and negative opinions, and evaluations within text. The reason for employing sentiment analysis in addition to our inductive dictionary design is to understand the emotion portrayed about each policy actor. One core assumption of the NPF is that “meaningful parts of policy reality are socially constructed” (Shanahan et al., 2018, p. 333). To analyze this social assignment of meaning to actors, objects, and processes in an automated fashion, an appropriate method needs to include evaluation of the words and phrases constructing the subject. Sentiment analysis addresses the socially constructed aspect of character interpretation by evaluating the context around a character or policy actor.

      Although there are numerous sentiment analysis programs available, this analysis uses the psychology-oriented Harvard-IV dictionary as used in the General Inquirer software, which is a simple polarity analysis using a dictionary of words associated with negative (2,005 words) or positive (1,637 words) (Stone, 2018). These words are oriented towards psychological concepts, as well as evaluations such as important, entrepreneurial, and failure. The Sentiment Analysis package for R allows for the analysis of sentiment of a single word up to a whole article (this study assigned sentiment scores to sentences). The result of sentiment analysis on a segment of text with the General Inquirer dictionary yields a score that is between a negative pole (-1) and a positive pole (1). For example, a sentiment score of 0.133 (0.200 positivity minus 0.067 negativity) was assigned to this sentence: Paul’s light protests have been part of his and the Boulder County Protectors’ effort to stop the oil and gas industry from drilling in Boulder County (Arvenson, 2017).

      This approach evaluates the linguistic context of the sentences that characters are found in. If characters are most often nested in sentences with negative sentiment, whether it is actually a direct construction about that character or not, it is arguable that the sentiment of that sentence is passed to the reader and associated with the character. Additionally, threats to the internal validity when using a dictionary approach (e.g., words may be inappropriately scored based on their context or include negation terms) may be minimized by the addition of sentiment analysis. Sentiment analysis also accounts for a larger portion of the identified words in a sentence, rather than depending on the correct interpretation of one or two dictionary words.

2. Use descriptive statistics to investigate the distribution of the sentiment scores

      After running the text through sentiment analysis and storing values with their respective sentences (in a new dataframe column in R), the next step is to investigate the descriptive statistics of each category. FBasics is an R package that may be used to summarize descriptive statistics of the sentiment for each policy actor and character category within a frame. Visual analysis using probability distribution functions allows the researcher to decide which measures of central tendency are most appropriate for comparison and whether statistical methods designed for normal distributions will be appropriate for application.

      Distributions may very well peak off the central sentiment value of 0.0 or be heavily skewed, yet still be representative. However, if they are multimodal—which implies the presence of several populations—central distribution values may be misleading. The presence of multiple populations may indicate a shift in sentiment over time, poorly constructed dictionaries, or representation of multiple communities (e.g., mixing news sources that may construct policy actors differently). If multimodal distributions are present, consider investigating this as part of the analysis as it may yield increased nuance in your results.

3. Relate measures of central tendency and hypothesis testing in order to “map” policy actors onto NPF characters

      To map the NPF characters to policy actors as would be the purpose in a manual NPF coding, Welch two-sample t-tests will establish whether coded villains, victims, and heroes (or, additionally, allies, opponents, and experts as we use in this study) hold significant relationships with particular policy actors. When NPF characters occupy a discrete space in the range of sentiment distributions, policy actor sentiment scores from the same group of articles may be mapped to them with some confidence. Based on this concept, this method identifies similarities of sentiment between policy actors and NPF characters by identifying those that hold the same sentiment (Welch t-tests that show an acceptance of the null hypothesis).7 To demonstrate the mapping method on individual frames as well as analytical interpretation techniques, we walk through examples in the Results section of this study.

Method Premise

     We use the above three steps to briefly demonstrate the statistical premise for the application of this method. Although the general method suggests subsetting to only one frame, we conduct this part of the statistical analysis across all frames to demonstrate that each NPF character’s distribution of sentiment scores occupies a distinct space that can be used to “map” onto the sentiment scores of policy actors. An investigation into the sentiment distributions of segments coded with NPF characters shows that most are relatively normal distributions, though allies, experts, and opponents show slightly bimodal distributions—multiple bumps rather than one hump (Figure 3). Across frames, multimodality is not totally unexpected for these three characters because they may have slightly varying constructions in different frames. In contrast, heroes, villains, and victims are strongly unimodal likely because they have more universally consistent constructions.

            If the sentiment distribution of scores of each NPF character is significantly different from each other, they may be considered distinct. Welch two-sided t-tests performed on each character category of sentiment scores relative to every other character demonstrate that NPF characters have statistically different distributions of the sentiment, except for heroes and experts above the 90% confidence level (Table 4). Considering the commitment of journalists to report with low levels of bias, it is conceivable that experts may be equivalent to heroes in news policy narratives, particularly when it comes to more technical, less value-driven frames, like those about wastewater injection or air quality in hydraulic fracturing for shale gas. Although the table shows a clear distinction between categories, within one frame the separation between characters has the potential to be even more clearly defined.

^{Note:                              =p<0.1; *p<0.05; **p<0.01; ***p<0.001}

       Some of these categories co-occurred so often they could potentially be considered as aspects of the same issue frame. After autocoding with a refined version of the frame dictionary, we ran a correlation and clustering analysis to investigate the distinctness of our dictionary categories. This method uses the tagged category counts, representing issue frames, at the article level. We found a high degree of positive correlation between a number of issue frames (Figure 4) suggesting that we could combine several of these issue categories. For instance, wastewater disposal and induced seismicity are correlated, as are politics and state fiscal, climate change and air quality, and local nuisance and traffic. However, for the results presented here, we decided to keep these issue frames separate to add a more nuanced investigation into the relatedness of aspects of issues. Figure 4 also reveals the distinctness of inversely-correlated frames (implying that when one frame increases the other decreases), such as jobs and economy and authority, politics and water quality, and wastewater disposal and state fiscal.

 

Figure 4. Correlation and clustering analysis of frames in autocoded articles

Solution Dictionary

           An initial solution dictionary was designed using the NPF concept of “moral of the story”—a policy solution that may end in a call to action (Shanahan et al., 2018). As with the policy referent dictionary, which contains some similarly derived terms, the solution dictionary may include general steps to achieving a policy goal, a stated policy preference, or named policies. To reduce the complexity posed by the large number of proposed solutions in the news articles, we employed a categorical coding design by policy tool (e.g., regulatory, subsidies, taxation). Though this categorization focused on proposed government solutions, we added a category for those that were oriented towards citizen action. To capture general statements of solutions to policy problems, we created the category, general solutions. Identified general solutions included phrases that suggest the need for a solution. For example, statements including a modal auxiliary (such as need to, should), indicated that they are proposing solutions. Additionally, phrases about the future (i.e., for our future), superlatives (i.e., is better than), and verbs in the present tense (i.e., to improve) usually pointed to solution statements. The dictionary was supplemented with named policy solutions by generating lists of proper names and acronyms.

Subsetting, Mapping, and Analysis

      After a final autocoding, we subset coded segments into separate issue frame categories. After running the segments through sentiment analysis, the resulting data contains segments that are each associated with categorical tags and sentiment scores.

      We investigated the relationships between the frames and coded narrative elements using the mapping technique presented above as well as regression. Ordinary least squares (OLS) regression analysis was performed with the package lm in R in order to address the research question that asks how frames, characters, and solutions are related. There is no assumption of causality with the use of regression in this study, rather it is used to investigate the relationship between frame, character code, and solution code. The general theoretical relationship between variables is frame » f(characters + solutions). The regressions of each frame were calculated using frequency count matrices, which are large tables containing a row for each article with a column for each frame. Because more than one frame identifier can occur within one article, these counts are whole numbers equal to or greater than zero. Similarly, these large matrices were constructed for characters and solutions. The probability distributions of characters, solutions, and frames are each approximately normal. For these reasons, OLS is suitable for this analysis.

      Measures of central tendency, such as statistical mean, kurtosis, skew, and quartiles, were calculated for character and policy actor sentiment scores with the R package, BasicStats. We also analyzed the Welch two-sample t-tests for characters and policy actors with the base R function, t.test. These scores allow for mapping policy actors onto NPF characters for each frame.

Results

      The results of this analysis suggest that frames are often correlated with certain characters and solutions. Using OLS regression analysis, sentiment analysis, descriptive statistics, and hypothesis testing, we find relationships between frames and characters and solutions. We also relate certain policy actors to NPF characters. The general form of the regression is FRAME ~ β0 + β1CHAR1 + β2CHAR2…+ β3SOLN1+ β4SOLN2 + …

       To investigate the relationship of frames with solutions and characters, we first calculated OLS regression statistics of a single frame against all solutions and characters using frequency count matrices. As a second step, we evaluate the sentiment associated with characters and policy actors to show how policy actors relate to NPF characters in this semi-automated method of textual analysis. The results of this investigation show that all frames have significant relationships to characters and solutions (Table 7). We briefly discuss several examples of significant findings from all those regressions (see Appendix A for regressions).

          An interesting result we mentioned in the General Method Description section, is that the NPF characters hero and experts hold a similar overall sentiment signature, whereas all other NPF characters have a significantly different sentiment distribution (Table 4). This finding suggests that the heroes of news articles may be experts, potentially influenced by the journalistic norm of presenting unbiased information. In the examples presented below for individual frames, we analyzed these characters separately, as they are discussed in NPF literature.

Frame: State Fiscal

       An example of a finding that suggests high predictability of elements in a certain frame is the state fiscal frame. The predictors explain much of the variance of the issue frame (R2adj =0.777, F(7,5696)=2,834.88, p<0.001). The government policy actor codes (significant above the 99.9% confidence level) were relatively predictive of the frame, suggesting that policy narratives containing frames about state budgets have a predictable formula, even across six state and local papers. Based on an understanding of how news articles are constructed, it is highly unlikely that journalists start with a policy actor and then create a frame. The interpretation is of co-occurrence, but not of causation (i.e., the independent variables of characters and solutions likely did not determine the chosen frame (the dependent variable)). The significant policy actors in the state fiscal frame were found to be activist group, general govt, local government, and named people (categories ending in SP in Table 7) at the state and local levels. However, the coefficients in this regression suggest that as the frequency of some actors increase in a narrative, the frame terms decrease. For example, the coefficients (regression results in Appendix B) for activist group and local government (βACTVISTGROUP=-0.296, βLOCALGOVT = -0.065) have an inverse relationship with this frame. Therefore, we focus on those with positive coefficients.

Table 7. Regression Variables and Significance

^{Note:                             +p<0.1; * p<0.05; ** p<0.01; *** p<0.001}

           As a first step in relating the policy actors to NPF characters, sentiment analysis was conducted on character-containing sentences within articles that have a state fiscal frame. From this process, each sentence containing a character was associated with a sentiment score. The resulting distributions with mean and 1st and 3rd quartiles are shown in Figure 5. The boxplots show that in the articles that contain this frame, the named local government official has the lowest sentiment (MGOVTLOCALSP=0.0450) and named state government official (MGOVTSTATESP=0.0628) has the highest. Comparatively, the values for the theoretical characters tagged in these articles have a low range suggesting that there is very little polarization in these articles. An interesting finding in this work and Crow and Wolton (2020) is that allies have a higher sentiment than heroes (MHERO=0.0700< MALLIES=0.0927). This is a consistent finding in sentiment across all the articles and many other frames as well. Policy actors are not as villainized in the state fiscal frame as they are in the whole set of articles as evidenced by the mean sentiment that is closer to zero (MVILLAIN_ALL=-0.0401< MVILLAIN_FISCAL=-0.0172). These findings suggest there is less polarization and overt depiction of heroes and villains in this local frame.

 

Figure 5.  Sentiment distributions for significant characters

      The second step in relating policy actors to NPF characters was hypothesis testing of the significant characters against the NPF characters (Table 8). Welch two-sample, two-sided t-tests were performed between the sentiment distributions of each significant actor and the NPF characters from the state fiscal frame. This maps the significant actors in a frame, identified at the sentence level with dictionary tagging, to the closest NPF character. Some actors will not be significantly associated with an NPF character in every frame. The hypothesis testing relies on the rejection of the null hypothesis in the case of testing the true difference in means, meaning that the populations cannot be interpreted as significantly different (on the chart this is marked by a No—we cannot reject the null—for significance. Table 8 shows that named state government, government, and named local government actors are most similar to the hero sentiment population, with the named state government actor having the most significant relationship. In this frame, these government actors are associated with moderately constructed heroes, while there are no clear actor associations with other characters.

Table 8. T-test results on significant actors against NPF characters

---

µ1 – µ2	Mean difference	Std. Error	t-statistic	Pr(>|t|)	Similar sentiment
GOVT STATE SP – HERO	-0.007	0.016	-0.452	0.652	Yes
GOVT – HERO	-0.008	0.016	-0.486	0.629	Yes
GOVT LOCAL SP – HERO	-0.025	0.017	-1.506	0.136	Yes
GOVT LOCAL SP – VICTIM	0.063	0.022	2.847	0.006**	No
GOVT – VICTIM	0.081	0.022	3.716	0.000***	No
GOVT STATE SP – VICTIM	0.081	0.022	3.717	0.000***	No
GOVT LOCAL SP – VILLAIN	0.062	0.014	4.490	0.000***	No
GOVT STATE SP – VILLAIN	0.080	0.013	6.076	0.000***	No
GOVT – VILLAIN	0.080	0.013	6.136	0.000***	No
GOVT STATE SP – EXPERT	-0.025	0.004	-5.880	0.000***	No
GOVT LOCAL SP – EXPERT	-0.043	0.006	-7.091	0.000***	No
GOVT – EXPERT	-0.026	0.004	-7.084	0.000***	No
GOVT STATE SP – ALLIES	-0.036	0.005	-7.684	0.000***	No
GOVT LOCAL SP – ALLIES	-0.054	0.006	-8.459	0.000***	No
GOVT – ALLIES	-0.036	0.004	-8.958	0.000***	No
GOVT LOCAL SP – OPPONENT	-0.051	0.005	-9.346	0.000***	No
GOVT STATE SP – OPPONENT	-0.033	0.003	-9.960	0.000***	No
GOVT – OPPONENT	-0.033	0.002	-14.175	0.000***	No

^{Note:                             +p<0.1; * p<0.05; ** p<0.01; *** p<0.001}

      The solutions highly related to the state fiscal frame include tax, regulation, and general solutions based on regression coefficients (βTAX=1.875, βREGULATION = -0.128, βGENERAL = -0.059). However, while the positive coefficient for tax predicts the co-occurrence of the state fiscal frame, both regulation and general solutions have a relatively strong negative relationship with this frame. This relationship implies that as the occurrence of the regulation and general solutions decrease and the tax solution increases, the likelihood of the frame being state fiscal increases. Although the regression coefficient implies positivity about the tax solution, the coding method used here is not evaluative of whether the solution is accepted. The support for solutions in the coded sentences can vary:

Mr. Rendell’s prediction that the tax would bring in $1.7 million in its first year, Mr. Rhoads said, is “dead wrong.” (Eagle, 2009)

Oil and gas companies, however, argue that the tax will have the opposite effect, harming Pennsylvanians by destroying the opportunity for thousands of well-paying jobs. (Eagle, 2009)

While state legislators continue to debate the need for a tax on natural gas drilling, proponents of the tax yesterday concerned themselves with where the revenue should go. (Eagle, 2009)

As previously shown in Figure 4, the state fiscal frame is moderately correlated (0.3) with the political frame and mildly anti-correlated (-0.09) with the authority frame, a result supported by the coefficient findings for tax and regulations. The state fiscal frame was found in 1,937 articles—though not necessarily the dominant frame—indicating that is an important theme in the discussion of shale development in the newspapers evaluated. The high occurrence of this frame category and the high coefficient of determination associated with the use of these actors and solutions suggest a similarity of the frame and the use of actors and solutions across states and over the 10-year period.

Frame: Wastewater Disposal

       The OLS regression results also indicate that some issues are associated with a larger number of narrative elements, including NPF characters. The wastewater disposal issue, which is highly correlated (0.51) with induced seismicity and moderately correlated with water quality (0.28) and water use (0.19), contains significant relationships with the NPF characters of villains, victims, and experts. Three solutions variables (information, permit, and named regulation) and actors (state government, experts, government, government local, government federal, and villain) mildly explain the variance of the issue frame at the 99.9% confidence level (R2adj =0.082, F(9,5698)=56.514, p<0.001). The strong positive correlation of the wastewater disposal frame to other frames (see Figure 4) suggests that there are various ways of talking about this issue. Comparing this frame to the relatively compact state fiscal frame suggests that some frames are associated with greater variability in character use.

       Additionally, the regression results show the use of more NPF characters, which indicate more polarization in these narratives. In fact, the NPF villain character has the highest coefficient of characters that explain variability (βVILLAIN=0.227, βEXPERT = 0.163, βSTATEGOVT = 0.106, βFEDGOVT=0.064, βGOVT = -0.032, βLOCALGOVT = -0.051) at the 99.9% confidence level. The sentiment scores for the characters support the idea of increased narrativity of wastewater disposal policy narratives. Sentiment analysis was conducted on character-containing sentences within articles that have a wastewater disposal frame. The results are shown in Figure 6. In the articles that contain this frame, the figure shows there is often a relatively negative villain compared to the villains in all of the articles (MVILLAIN_ALL= -0.0261 > MVILLAIN_WWD= -0.0485).

        Yet, at the 99.9% significance level, villains were not associated with the industry or opponent actor, which is significant to the model at the 95% confidence level. Interestingly, villains in these narratives are not highly associated with any particular policy actor. A partial explanation may be that policy actors are reluctant to name industry, which is constructed very positively in the dataset (MINDUSTRY_ALL= 0.1020) and has a similar construction to the character allies (MINDUSTRY_ALL= 0.1020 ~ MALLIES_ALL= 0.1083). Examples of segments containing the villain character show the reluctance for villainization of the industry that is producing the wastewater but often portray the haulers of the waste as deviant individuals:

Local officials in the Oil Patch say they know some truck drivers are dumping liquid waste in remote areas to save time and money, but it’s rare to be able to pursue criminal charges. (Dalrymple, 2014)

Mckenzie County, which has the highest concentration of drilling activity, has not yet had any criminal cases for illegal dumping, yet Samuelson estimates he receives two to three reports a month. (Dalrymple, 2014)

A Western Pennsylvania waste-hauler, Robert Allan Shipman, was convicted of illegally dumping waste in 2012, and sentenced to serve seven years of probation and 1,750 hours of community service, and to pay $382,000 in restitution and fines. The attorney general has appealed the sentence, arguing that Shipman deserved jail time. (Maykuth, 2013)

Nathan Garber of Kalispell, Mont., is accused of ordering the illegal dumping of saltwater waste into a former oil well, putting drinking water in Stark County at risk for contamination. He faces up to five years in prison and a $5,000 fine if convicted of the felony. (Associated Press, 2012)

     Figure 6 shows that of the most positively constructed character group in this frame is state government actors, which are constructed most closely with allies (MALLIES=0.1027< MSTATEGOVT=0.1197). Again, allies have a higher sentiment than heroes (MHERO=0.0976< MALLIES=0.1027).

Figure 6. Significant policy actors and NPF characters in the wastewater disposal frame

        Among the articles using this frame, there were few references to victim (n=10) or hero (n=22), which prevented hypothesis testing, though victims were significantly related to this frame in the regression at the 99% confidence level (i.e., when it is used, this is one of the frames it is highly associated with). The results in Table 6 showed that local government and government are most associated with experts and that state government and industry are most associated with allies.

       The solutions presented in this frame of highest significance (99.9% confidence level) are named regulations, permit, and information. Based on an evaluation of the regression coefficients (βNAMEDREGULATION = 0.243, βPERMIT = 0.104, βINFORMATION = -0.129), named regulation is the solution co-occurring most often. When investigated, these were found to be environmentally-related laws (e.g., Clean Streams Law, Ground Water Protection Act, Clean Water Act). While the positive coefficients for named regulation and permit show the prediction of the co-occurrence of the wastewater disposal frame, information solutions have a relatively strong negative relationship with this frame. Because it is negative, the regression coefficient implies that as the occurrence of information solutions increase, the likelihood of the frame being wastewater disposal decreases. Negative coefficients can occur in the regression results because the regression was done across all frames, characters, and solutions.

Table 9. T-test results on significant actors against NPF characters

---

µ1-µ2	Mean difference	Std. Error	t-stat	Pr(>|t|)	Similar sentiment
GOVT  LOCAL – EXPERT	-0.001	0.006	-0.198	0.843	Yes
GOVT – EXPERT	-0.002	0.005	-0.479	0.632	Yes
GOVT  STATE – ALLIES	0.006	0.008	0.740	0.460	Yes
INDUSTRY – ALLIES	-0.006	0.008	-0.745	0.457	Yes
GOVT  LOCAL – OPPONENT	-0.013	0.006	-2.121	0.034*	No
GOVT  FED – EXPERT	-0.012	0.005	-2.178	0.030*	No
GOVT – OPPONENT	-0.014	0.005	-2.868	0.004**	No
GOVT  LOCAL – ALLIES	-0.036	0.009	-3.812	0.000***	No
GOVT – ALLIES	-0.037	0.009	-4.266	0.000***	No
INDUSTRY – OPPONENT	0.017	0.004	4.302	0.000***	No
GOVT  FED – OPPONENT	-0.024	0.005	-4.524	0.000***	No
GOVT  FED – ALLIES	-0.047	0.009	-5.250	0.000***	No
GOVT  STATE – OPPONENT	0.029	0.004	6.583	0.000***	No
INDUSTRY – EXPERT	0.029	0.004	6.845	0.000***	No
GOVT  FED – VILLAIN	0.126	0.013	9.390	0.000***	No
GOVT  STATE – EXPERT	0.041	0.005	8.775	0.000***	No
GOVT  LOCAL – EXPERT	0.136	0.014	9.888	0.000***	No
GOVT – VILLAIN	0.135	0.013	10.156	0.000***	No
INDUSTRY – VILLAIN	0.166	0.013	12.850	0.000***	No
GOVT  STATE – VILLAIN	0.179	0.013	13.631	0.000***	No

       Villains in the wastewater disposal issue frame have lower than average negative sentiment and state government are allies with greater than average positive sentiment, suggesting this is a contentious issue. Interestingly, named regulations are frequently associated with this topic, while information is a solution rarely posed. Likely, when policy actors are characterized as villains, we don’t imagine that the soft policy tool of information would make an impact in the situation.  Here, state government is seen as an ally in the fight against dumping using existing laws and regulations to solve the problem.

Discussion

       One of the main objectives of this chapter has been to investigate the relationship between frames, characters, and solutions. Our results show significant relationships between frames and those narrative elements, confirming some definitional assumptions about frames that had not, up to this point, been investigated. Here we have found aspects of Entman’s (1993) definition of frames to hold empirical validity: frames narrow our lens by promoting particular presentations of reality as well as treatment recommendations. Our findings suggest that not only are policy actors often closely tied to frames, but they are also accompanied by specific policy solutions. These findings have implications for advancing our understanding of the structure that frames can impose on a narrative, including elements such as characters and solutions.

        While we only investigated a small portion of the characters, solutions, and frames in the hydraulic fracturing analysis, the application of this method provided us with findings about the narratives of the issue in the six state and six local newspapers as well as policy narratives in general. The state fiscal frame has a very high predictability of characters and solutions, but has a very low range of sentiment, marking it as a consistently used, yet uncontentious issue frame of hydraulic fracturing. These findings suggest that frames can carry consistent elements regardless of their level of NPF characterization. More contentious issue frames, like wastewater disposal, involve an increased number of NPF characterizations that are constructed with a wider range of sentiment than others. Importantly to the advancement of the NPF, this method provides a quantitative measure of the relative level of good or bad associated with a character. The relative level of characterization clues the researcher into the aspects of issues that are the thorniest and can potentially highlight the solutions that may be acceptable to the public.

       We have demonstrated that NPF research will benefit from including frames in a standard analysis because they allow for advances in understanding narrative structure and when characters are expected to be associated with particular solutions. Research on large datasets of policy narratives could increase our understanding of other factors that likely impact characters and solutions, such as urgency or proximity of the problem. The use of sentiment analysis in conjunction with inductive methods may also lead us to insights on devil-shift or angel-shift patterns. NPF research could use the relationship between frames and policy design we have demonstrated here to address scope of conflict strategies in an automated fashion as well, particularly since we found that policy actors are associated with certain designs that hold relatively fixed cost/benefit distributions.

      Gaining a deeper understanding of how frames bound the presence of characters and solutions in narratives will contribute to a stronger tie between the framing and NPF literature. With our initial findings pointing to a relationship between frame and narrative elements such as characters and solutions, we recognize the potential for some shared properties of NPF plots and frames. For instance, frames may act in similar ways to an NPF plot, “situat[ing] the characters and their relationship in time and space” (Shanahan et al., 2017, p. 175). More deeply understanding the overlap between plots and frames may provide routes to increased consistency in the coding of plots, which has historically been challenging.

       As researchers consider using semi-automated methods, they must also consider the tradeoffs. Automated textual analysis, even with the use of iteratively refined dictionaries, may not detect some nuances of language. In some cases, researchers may consider this an acceptable compromise with the increased generalizability of the findings from having a larger dataset and the gains in reliability from automation. Additionally, while these methods appear to be faster because they are automated, they take a similarly deep consideration of research design as do hand-coding studies. Significant time must be invested in conceptualizing, populating, checking the validity of, and refining dictionaries for this process. And if one is a novice programmer, time must be spent in understanding terminology, data structures, and available software.

         The narrative research in this study was only a first step towards understanding automated tools that may increase the ability to address larger datasets systematically. Our research has limitations that could be addressed in future research. For instance, while context around characters is important, is a whole sentence a good predictor of emotional response about one paragraph? Perhaps it is only in certain parts of speech in that sentence, as suggested by Shanahan et al. (2018), or can it be contained in one evaluative word? We also make assumptions in our theoretical definitions that could be refined by future research. For instance, although we did use some entities as policy actors, such as industry and levels of government, we did not include non-human characters and abstract principles as entities. For example, although we did not treat it as such, the environment could be constructed as a victim or greenhouse gas emissions could be a villain in a frame about climate change.

       The results from this research have implications for the way that policy narrative researchers do business. While this chapter only discusses several of the frames, policy actors/characters, and solutions, it should be clear that these methods add autocoding and computer-assisted tools to the policy narrative researcher’s toolkit. Additionally, advances in computation linguistics research suggest there are more automated tools to come. For instance, Jacobs (2019) shows the potential for vector space models aided by sentiment analysis to predict the emotional potential of fictional characters—a method that could aid NPF researchers, particularly in the realm of character identification with large volumes of text. While there are limitations to automated methods, we have shown that an iterative context-considered approach can open up the potential for addressing larger, more complex datasets with relatively high levels of confidence. The iterative, inductive approach we have presented also means that researchers hold greater levels of understanding of their own methods and results.

References

Adgate, John L., Bernard D. Goldstein, and Lisa M. McKenzie. 2014. “Potential Public Health Hazards, Exposures and Health Effects from Unconventional Natural Gas Development.” Environmental Science & Technology 48 (15): 8307-8320. https://doi.org/10.1021/es404621d.

Arvenson, Amelia. 2017. “David Paul Pleads not Guilty in Case of Beaming Light onto Boulder Courthouse.” Longmont Times Call, December 11, 2017. https://www.timescall.com/2017/12/11/david-paul-pleads-not-guilty-in-case-of-beaming-light-onto-boulder-courthouse/.

Associated Press. 2012. “Mont. Oil Executive Pleads not Guilty to Crime in N.D.” Bismarck Tribune, November 28, 2012.

Bamberger, Michelle and Robert E. Oswald. 2015. “Long-term Impacts of Unconventional Drilling Operations on Human and Animal Health.” Journal of Environmental Science and Health, Part A 50 (5): 447-459. https://doi.org/10.1080/10934529.2015.992655.

Berardo, Ramiro, Federico Holm, Tanya Heikkila, Christopher M. Weible, Hongtao Yi, Jennifer Kagan, Catherine Chen, and Jill Yordy. 2020. “Hydraulic Fracturing and Political Conflict: News Media Coverage of Topics and Themes across Nine States.” Energy Research & Social Science 70: 101660. https://doi.org/10.1016/j.erss.2020.101660.

Blair, Benjamin, Tanya Heikkila, and Christopher M. Weible. 2016. “National Media Coverage of Hydraulic Fracturing in the United States: Evaluation Using Human and Automated Coding Techniques: Hydraulic Fracturing Media Coverage.” Risk, Hazards & Crisis in Public Policy 7 (3): 114-128. https://doi.org/10.1002/rhc3.12097.

Chong, Dennis and James N. Druckman. 2007. “Framing Theory.” Annual Review of Political Science 10 (1): 103-126. https://doi.org/10.1146/annurev.polisci.10.072805.103054.

Crow, Deserai A., Lydia A. Lawhon, John Berggren, Juhi Huda, Elizabeth Koebele, and Adrianne Kroepsch. 2017. “A Narrative Policy Framework Analysis of Wildfire Policy Discussions in Two Colorado Communities.” Politics & Policy 45 (4): 626-656. https://doi.org/10.1111/polp.12207.

Crow, Deserai A., and Andrea Lawlor. 2016. “Media in the Policy Process: Using Framing and Narratives to Understand Policy Influences.” Review of Policy Research 33 (5): 472-491. https://doi.org/10.1111/ropr.12187.

Crow, Deserai A. and Laura Wolton. 2020. “Talking Policy in Congressional Campaigns: Construction of Policy Narratives in Electoral Politics.” Politics & Policy 48 (4): 658-699.
https://doi.org/10.1111/polp.12369.

Dalrymple, Amy. 2014. “Truckers Dumping Illegal Waste Rarely Caught.” Grand Forks Herald, February 6, 2014.

Eagle, Jess. 2009. “Supporters of Gas Drilling Tax Suggest How it Should be Spent.” Pittsburgh Post Gazette, June 13, 2009.

Entman, Robert M. 1993. “Framing: Toward Clarification of a Fractured Paradigm.” Journal of Communication 43 (4): 51-58. https://doi.org/10.1111/j.1460-2466.1993.tb01304.x.

Gallegos, Tanya J., Brian A. Varela, Seth S. Haines, and Mark A. Engle. 2015. “Hydraulic Fracturing Water Use Variability in the United States and Potential Environmental Implications.” Water Resources Research 51 (7): 5839-5845. https://doi.org/10.1002/2015WR017278.

Hand, Eric. 2015. “Oil and Gas Operations Could Trigger Large Earthquakes.” Science, April 23, 2015. https://www.sciencemag.org/news/2015/04/oil-and-gas-operations-could-trigger-large-earthquakes.

Heikkila, Tanya, and Christopher M. Weible. 2017. “Unpacking the Intensity of Policy Conflict: a Study of Colorado’s Oil and Gas Subsystem.” Policy Sciences 50 (2): 179. https://doi.org/10.1007/s11077-017-9285-1.

Humphreys, Ashlee, and Rebecca Jen-Hui Wang. 2017. “Automated Text Analysis for Consumer Research.” Journal of Consumer Research 44 (6): 1274-1306. https://doi.org/10.1093/jcr/ucx104.

Jacobi, Carina, Wouter van Atteveldt, and Kasper Welbers. 2015. “Quantitative Analysis of Large Amounts of Journalistic Texts Using Topic Modelling.” Digital Journalism 4 (1): 89-106.
https://doi.org/10.1080/21670811.2015.1093271.

Jacobs, Arthur M. 2019. “Sentiment Analysis for Words and Fiction Characters From the Perspective of Computational (Neuro-)Poetics.” Frontiers in Robotics and AI 6 (53). https://doi.org/10.3389/frobt.2019.00053.

Jones, Michael D. 2013. “Cultural Characters and Climate Change: How Heroes Shape Our Perception of Climate Science.” Social Science Quarterly 95 (1): 1-39.

Jones, Michael D. and Mark K. McBeth. 2010. “A Narrative Policy Framework: Clear Enough to Be Wrong?” Policy Studies Journal 38 (2): 329.

Jones, Michael D., Elizabeth A. Shanahan, and Mark K. McBeth. 2014. The Science of Stories: Applications of the Narrative Policy Framework in Public Policy Analysis.  London: Palgrave MacMillan.

Jones, Michael D. and Geoboo Song. 2014. “Making Sense of Climate Change: How Story Frames Shape Cognition.” Political Psychology 35 (4): 447-476. https://doi.org/10.1111/pops.12057.

Joyce, Stephanie and Jordan Wirfs-Brock. “The Rising Cost of Cleaning Up After Oil and Gas.” Inside Energy, October 1, 2015.

Konkel, Lindsey. 2016. “Salting the Earth: The Environmental Impact of Oil and Gas Wastewater Spills.” Environmental Health Perspectives 124 (12): A230-A235. https://doi.org/doi:10.1289/ehp.124-A230.

Krippendorff, Klaus. 2004. Content Analysis: An Introduction to its Methodology. Thousand Oaks, Calif: Sage.

Lawlor, Andrea. 2015. “Framing Immigration in the Canadian and British News Media.” Canadian Journal of Political Science 48 (2): 329. https://doi.org/10.1017/S0008423915000499.

Lawlor, Andrea and Deserai A. Crow. 2018. “Risk-Based Policy Narratives.” Policy Studies Journal 46 (4): 843-867. https://doi.org/843-867.

Lawton, Ricky N. and Murray A. Rudd. 2014. “A Narrative Policy Approach to Environmental Conservation.” Ambio 43 (7): 849-857. https://doi.org/10.1007/s13280-014-0497-8.

Mason, Krystal L., Kyla D. Retzer, Ryan Hill, and Jennifer M. Lincoln. 2015. “Occupational Fatalities During the Oil and Gas Boom — United States, 2003 – 2013.” Morbidity and Mortality Weekly Report (MMWR) 64 (20): 551-554.

Maykuth, Andrew. 2013. “Shale Criminal Charges Stun Drilling Industry.” Philadelphia Inquirer, September 12, 2013.

McGinnis, Michael D. and Elinor Ostrom. 2014. “Social-ecological System Framework: Initial Changes and Continuing Challenges.” Ecology and Society 19 (2): 30. https://doi.org/10.5751/es-06387-190230.

Merry, Melissa. 2016. “Making Friends and Enemies on Social Media: The Case of Gun Policy Organizations.” Online Information Review 40 (5): 624-642.
https://doi.org/10.1108/OIR-10-2015-0333.

Merry, Melissa K. 2018. “Narrative Strategies in the Gun Policy Debate: Exploring Proximity and Social Construction.” Policy Studies Journal 46 (4): 747-770. https://doi.org/10.1111/psj.12255.

Miles, Matthew B., A. Michael Huberman, and Johnny Saldaña. 2020. Qualitative Data Analysis: A Methods Sourcebook. Thousand Oaks: Sage Publications, Inc.

Moskowitz, Peter. 2015. “New Report Estimates Enough Natural Gas is Leaking to Negate Climate Benefits.” The Guardian, June 24, 2015. http://www.theguardian.com/environment/2015/jun/24/natural-gas-leaks-methane-environment.

Olofsson, Kristin L., Christopher M. Weible, Tanya Heikkila, and J. C. Martel. 2018. “Using Nonprofit Narratives and News Media Framing to Depict Air Pollution in Delhi, India.” Environmental Communication 12 (7): 956-972. https://doi.org/10.1080/17524032.2017.1309442.

Poirier, William, Catherine Ouellet, Marc-Antoine Rancourt, Justine Béchard, and Yannick Dufresne. 2020. “(Un)Covering the COVID-19 Pandemic: Framing Analysis of the Crisis in Canada.” Canadian Journal of Political Science 53 (2): 365-371. https://doi.org/10.1017/S0008423920000372.

Ratner, Bruce. 2009. “The Correlation Coefficient: Its Values Range Between +1/−1, or Do They?” Journal of Targeting, Measurement and Analysis for Marketing 17 (2): 139-142. https://doi.org/10.1057/jt.2009.5.

Scheufele, Dietram A. and Shanto Iyengar. 2014. “The State of Framing Research: A Call for New Directions.” In The Oxford Handbook of Political Communication, edited by Kate Kenski and Kathleen Hall Jamieson. New York: Oxford University Press.

Schäfer, Mike S. and Saffron O’Neill. 2017. “Frame Analysis in Climate Change Communication: Approaches for Assessing Journalists’ Minds, Online Communication and Media Portrayals.” In Oxford Encyclopedia of Climate Change Communication, edited by Matthew Nisbet, Shirley Ho, Ezra Markowitz, Saffron O’Neill, Mike S. Schäfer and Jagadish Thaker. New York: Oxford University Press.

Scott, Tyler A., Nicola Ulibarri, and Ryan P. Scott. 2020. “Stakeholder Involvement in Collaborative Regulatory Processes: Using Automated Coding to Track Attendance and Actions.” Regulation & Governance 14 (2): 219-237. https://doi.org/10.1111/rego.12199.

Shanahan, Elizabeth A., Michael D. Jones, and Mark K. McBeth. 2018. “How to Conduct a Narrative Policy Framework Study.” The Social Science Journal 55 (3): 332-345. https://doi.org/10.1016/j.soscij.2017.12.002.

Shanahan, Elizabeth A., Michael D. Jones, Mark K. McBeth, and Claudio M. Radaelli. 2017. “The Narrative Policy Framework.” In Theories of the Policy Process, edited by Christopher M. Weible and Paul A. Sabatier. Boulder, CO: Westview.

Stone, Philip J., Robert F. Bales, J. Zvi Namenwirth, and Daniel M. Ogilvie. 1962. “The General Inquirer: A Computer System for Content Analysis and Retrieval Based on the Sentence as a Unit of Information.” Behavioral Science 7 (4): 484.

Stone, Phillip. 2018. “Descriptions of Inquirer Categories and Use of Inquirer Dictionaries.” Accessed December 8, 2018. http://www.wjh.harvard.edu/~inquirer/homecat.htm.

U.S. Energy Information Administration. 2018. “How Much Shale Gas is Produced in the United States?” Accessed April 10, 2018. https://www.eia.gov/tools/faqs/faq.php?id=907&t=8.

Wines, Michael. 2013. “Gas Drilling Not Imminent, but Debate Roils N.Y. Region.” Pittsburgh Post-Gazette, January 13, 2013.

Witze, Alexandra. 2015. “Artificial Quakes Shake Oklahoma: Earthquakes Linked to Oil and Gas Operations Prompt Further Research into Human-induced Seismic Hazards.” Nature 520 (7548): 418.

Yu, Jingyuan, Yanqin Lu, and Juan Muñoz-Justicia. 2020. “Analyzing Spanish News Frames on Twitter during COVID-19—A Network Study of El País and El Mundo.” International Journal of Environmental Research and Public Health 17 (15): 5414. https://doi.org/10.3390/ijerph17155414.

Chapter Appendixes

Appendix A. Included news sources and article counts

---

Newspaper Name	Publishing City	State	Pre-exclusion Count	Source	Article count post-exclusion
Houston Chronicle	Houston	TX	2935	Newsbank	592
Dallas Morning News	Dallas	TX	1543	Newsbank	441
Pittsburgh Post-Gazette	Pittsburgh	PA	3992		1170
Philadelphia Inquirer	Philadelphia	PA	2397	Newsbank	691
Oklahoman	Oklahoma City	OK	2909		393
Tulsa World	Tulsa	OK	1868		187
The Advocate	Baton Rouge	LA	663	Newsbank	233
Times-Picayune	New Orleans	LA	682	Newsbank	170
Grand Forks Herald	Grand Forks	ND	1003	Newsbank	360
Bismarck Tribune	Bismarck	ND	2425	Newsbank	580
Denver Post	Denver	CO	1595	Newsbank	567
Daily Times-Call	Longmont	CO	1466	Newsbank	315
				Total Count	5708

Appendix B. Regression of significant characters and solutions at 99.9% confidence

 

Appendix C. Accepted interpretation of correlation coefficient

---

Correlation Coefficient Range	Relationship Interpretation (e.g., Ratner 2009)
0	no linear relationship
0 to 0.3 or 0 to -0.3	weak positive or negative relationship
0.3 to 0.7 or -0.3 to -0.7	moderate positive or negative relationship
0.7 to 1.0 or -0.7 to -1.0	strong positive or negative linear relationship
+1 or -1	perfect positive or negative relationship

 

Appendix D. Character dictionary

---

Category	Search item	Whole word	Case sensitivity	Starting letters
CHAR_ALLIES	advocacy group	0	0	0
CHAR_ALLIES	advocate	0	0	0
CHAR_ALLIES	backer	0	0	0
CHAR_ALLIES	campaigner	0	0	0
CHAR_ALLIES	champion	0	0	0
CHAR_ALLIES	counsel	0	0	0
CHAR_ALLIES	defender	0	0	0
CHAR_ALLIES	enthusiast	0	0	0
CHAR_ALLIES	friend	0	0	0
CHAR_ALLIES	guardian	0	0	0
CHAR_ALLIES	leader	0	0	0
CHAR_ALLIES	promoter	0	0	0
CHAR_ALLIES	proponent	0	0	0
CHAR_ALLIES	protector	0	0	0
CHAR_ALLIES	spokesperson	0	0	0
CHAR_ALLIES	supporter	0	0	0
CHAR_EXPERT	analyst	0	0	0
CHAR_EXPERT	chemist	0	0	0
CHAR_EXPERT	economist	0	0	0
CHAR_EXPERT	educator	0	0	0
CHAR_EXPERT	examiner	0	0	0
CHAR_EXPERT	expert	0	0	0
CHAR_EXPERT	geologist	1	0	0
CHAR_EXPERT	industry analyst	1	1	0
CHAR_EXPERT	inspector	0	0	0
CHAR_EXPERT	panelist	0	0	0
CHAR_EXPERT	physicist	0	0	0
CHAR_EXPERT	professor	0	0	0
CHAR_EXPERT	researcher	0	0	0
CHAR_EXPERT	scholar	0	0	0
CHAR_EXPERT	scientist	0	0	0
CHAR_EXPERT	seismologist	0	0	0
CHAR_EXPERT	specialist	0	0	0
CHAR_EXPERT	technician	0	0	0
CHAR_HERO	celebrity	0	0	0
CHAR_HERO	hero	0	0	0
CHAR_HERO	protagonist	0	0	0
CHAR_HERO	superstar	0	0	0
CHAR_HERO	winner	0	0	0
CHAR_OPPONENT	activist	0	0	0
CHAR_OPPONENT	adversary	0	0	0
CHAR_OPPONENT	agitator	0	0	0
CHAR_OPPONENT	antagonist	0	0	0
CHAR_OPPONENT	candidate	0	0	0
CHAR_OPPONENT	challenger	0	0	0
CHAR_OPPONENT	competitor	0	0	0
CHAR_OPPONENT	demonstrator	0	0	0
CHAR_OPPONENT	enem	0	0	1
CHAR_OPPONENT	foe	0	0	0
CHAR_OPPONENT	heckler	0	0	0
CHAR_OPPONENT	litigant	0	0	0
CHAR_OPPONENT	opponent	0	0	0
CHAR_OPPONENT	opposition	0	0	0
CHAR_OPPONENT	protester	0	0	0
CHAR_OPPONENT	radical	0	0	0
CHAR_OPPONENT	rival	0	0	0
CHAR_VICTIM	casualt	0	0	1
CHAR_VICTIM	fatalit	0	0	1
CHAR_VICTIM	injured part	0	0	1
CHAR_VICTIM	victim	0	0	0
CHAR_VILLAIN	convict	0	0	0
CHAR_VILLAIN	criminal	0	0	1
CHAR_VILLAIN	crook	0	0	0
CHAR_VILLAIN	delinquent	0	0	0
CHAR_VILLAIN	deviants	0	0	0
CHAR_VILLAIN	felon	1	0	0
CHAR_VILLAIN	guerrilla	0	0	0
CHAR_VILLAIN	guilty party	0	0	0
CHAR_VILLAIN	lawbreaker	0	0	0
CHAR_VILLAIN	perpetrator	0	0	1
CHAR_VILLAIN	rebel	0	0	0
CHAR_VILLAIN	suspect	0	0	0
CHAR_VILLAIN	terrorist	0	0	0
CHAR_VILLAIN	thug	0	0	0
CHAR_VILLAIN	villain	0	0	1

Appendix E. Autocoding example in R

---

The following lines of code are for use on a dataframe that contains source data and clean text.

The way this code is designed is that each row of the dataframe input text is segmented into the desired unit for the analysis. The following chunk of code breaks the text from a dataframe into sentences (the unit of analysis for this study). Here, we use the unnest_sentences function to change each whole block of text into a larger dataframe with the number of rows equal to the number of sentences in all the text. For example, if 100 news articles were originally read into a dataframe (text_df), the dimensions would be the column of text (text_df$text) plus columns for metadata (6 information columns shown below) by 100 (a row for each of the 100 articles). The dimension of the post-unnesting dataframe (text_sen) will be the number of sentences in all the documents.

library(lexRankr)
#text_df=dataframe with each cell in the “text” column contains the text
#from a whole document
dim(text_df)
#[1] 100   7
colnames(text_df)
#[1] “title”     “paper”     “date”      “section”   “byline”    “text”      “highlight”

#the column title contains each document name
#sents in the function call is a dataframe column that will be created containing the sentences
text_sen<-unnest_sentences(text_df,sents,text,doc_id=title)

#the number of rows in the new dataframe will equal the number of sentences
#in all the text documents
dim(text_sen)
#11220     8
colnames(text_sen)

#[1] “title”     “paper”     “date”      “section”   “byline”    “highlight” [7] “sent_id”  “sents”

For the dictionary to find all occurrences, the text must be cleaned. Remove all punctuation and multiple spaces.

#remove a little punctuation so the dictionary can find all occurrences
#this will create a new column of clean text in your dataframe (clntxt)
library(mgsub)
text_sen$clntxt<-mgsub(text_sen$sents, c(“[[:punct:]]”,”\r\n”), c(“”,” “))
text_sen$clntxt<-mgsub(text_sen$clntxt,“\\s+”,” “)

Read in the dictionary designed similarly to the one shown in Figure 1. Then change the values in the columns of switches to be grep keyword friendly.

library(readxl)
#read in frame dictionary
#this example is from a spreadsheet
#read in as a tibble or dataframe for compatibility with the following code
dict<-read_xlsx(‘C:/Users/vegan/Dropbox/Campaign Book Project/Dictionaries/pol_ref_dict.xlsx’,col_names=TRUE,col_types=NULL)

#rename cols
colnames(dict)[c(1:5)]<-c(“Category”,”word”,”ww”,”cs”,”sl”)

#make the columns friendly the grep search
#have to swap 0 to 1 and 1 to 0
#this is for the ignore.case keyword in grep
dict$cs<-replace(dict$cs,dict$cs==1,“FALSE”)
dict$cs<-replace(dict$cs,dict$cs==0,“TRUE”)

#this is for the starting letter
ind<-which(dict$sl==1)
#add a single space to the beginning of the word
dict$word[ind]<-paste(“”,dict$word[ind])
#remove any multiple spaces
dict$word<-mgsub(dict$word,“\\s+”,” “)

#if the word is the whole word add a space
#on the front and back of the word
ind=which(dict$ww==1)
dict$word[ind]<-paste(“”,dict$word[ind],””)

Create a new empty dataframe for the coded segments. The dimensions of the new dataframe will contain two additional columns for category and search term.

####create an empty dataframe to save the coded segments
#you’ll want to have enough columns to contain
#the original data info (e.g., title of article, source, date)
#plus columns that are needed for the category, search word
#ncol should be the number of columns in text_sen dataframe+2
coded_segs<-data.frame(matrix(ncol=10,nrow=0))

################
for (wind in 1:length(dict$word)){
#look for the word in tweets, tag it
ind<-grep(dict$word[wind],text_sen$clntxt,value=FALSE,ignore.case=dict$cs[wind])
#add to the output dataframe
coded_segs<-rbind(coded_segs,cbind(text_sen[ind,c(“date”,“paper”,“title”,“section”,“byline”,“sents”,“sent_id”)],cbind(cbind(rep(dict$Category[wind],length(ind)),rep(dict$word[wind],length(ind))),text_sen$clntxt[ind])))

}
colnames(coded_segs)<-c(colnames(text_sen)[c(1:6)],“Code”,“SearchWord”,“clntxt”)
coded_segs$clntxt<-as.character(coded_segs$clntxt)

The resulting dataframe of coded segments will contain a row for every tag. This dataframe can then be used for various analysis purposes.

 

[1] MAXQDA Plus or MAXQDA Analytics Pro are the versions of the software that contains MAXDictio plugin for a number of functions listed here.

[2] If the researcher chooses to code using general and subdividing categories containing overlapping search terms such as “government” and “local government”, the more general term tag may be dropped when a double-tagging occurs. In R, this may be accomplished using decision-making structures (i.e., if-else or case statements). However, this can often be avoided by increasing specificity of the search term such as using “the government” and “local government” rather than “government”, as was done in this study.

[3] Note that this method typically requires increased amounts of RAM to be able to analyze large text corpora. This study used 64GB of RAM.

[4] Typically, the minimum is considered to be counts above 30.

[5] This is generally considered to be n>30.

[6] For this phase, it is likely that the researcher will have to rely fully on R rather than on MAXQDA unless they are analyzing Tweets (MAXQDA does have a function for analyzing the sentiment of Tweets, but it will not assist with the mapping method described here).

[7] For Welch t-tests, the null hypothesis is that the true difference in means is equal to 0.0.

[8] The news articles were originally collected for a larger study on policy debates related to shale oil and gas development as described in (Berardo et al., 2020). Funding support for the larger study came from the National Science Foundation Award Foundation, Decision, Risk, and Management Sciences Program (grant SES-1734310 and SES-1734294).