Bright Line Watch Survey Report: Wave 6

Erosion, polarization, and norm violations: 
Bright Line Watch Survey Report
August 1, 2018 

Donald Trump is a disruptor. Few would disagree that his actions and style of political combat set him apart from past pres­i­dents. There is less consensus about the impli­ca­tions of Trump’s dis­tinc­tive governing style for the health of American democracy, however. 

To under­stand how Americans rate their democracy in the Trump era, Bright Line Watch (BLW) conducted its sixth expert survey, and its fourth public survey in July 2018. These consisted of an expert sample of 679 political science faculty at American uni­ver­si­ties (surveyed July 7–23) and a nation­al­ly rep­re­sen­ta­tive sample of 2,000 adults (surveyed July 11–18). Respondents in both surveys assessed the degree to which 27 demo­c­ra­t­ic prin­ci­ples (listed in an appendix to this report) are currently upheld in the United States and evaluated the overall quality of American democracy on a 100-point scale. Participants in our expert sample also rated a set of 22 state­ments describ­ing recent political events on two separate scales — how normal/abnormal they are in the context of U.S. politics, and how important/unimportant they are.  The second battery, on (ab)normality and (un)importance, is based on similar surveys conducted by the New York Times Upshot blog in February and May 2017.[1]

The key results from this survey are sobering:

  • American democracy is continuing to erode.  Our expert respondents perceive a consistent, ongoing decline in the overall quality of American democracy from 2015 (assessed retrospectively) to our first expert survey in February 2017 and over the five subsequent surveys we have conducted since then.

  • Public ratings of the quality of American democracy are continuing to polarize. Notably, those who disapprove of Trump report ongoing democratic erosion. However, those who approve of the president perceive the quality of democracy to have improved measurably during this period.

  • Across a range of measures, President Trump’s supporters and his detractors are drawing conclusions about democracy that are not merely disconnected, but are often antithetical to each other.

  • A number of recent political events are characterized by experts as both highly abnormal and important. 

Erosion in the quality of American democracy

We start by sum­ma­riz­ing expert eval­u­a­tions of the overall quality of American democracy through­out U.S. history and during the Trump era. Our May 2017 survey asked the expert sample to rate the quality of U.S. democracy currently and ret­ro­spec­tive­ly at nine his­tor­i­cal dates: 1800, 1850, 1900, 1950, 1975, 1985, 1995, 2005, and 2015. We have asked for the same rating in each of our six expert surveys (February, May, and September 2017; January, April, and July 2018). 

In the figure below, the black line with dots shows the average ratings from the Bright Line Watch expert sample for each his­tor­i­cal date plus the May 2017 and July 2018 ratings. It is measured on a 0–100 scale. For com­par­i­son, the grey line shows the Liberal Democracy Index from the Varieties of Democracy Project (V‑Dem), rescaled to the same 0–100 scale, which extends back to the rat­i­fi­ca­tion of the Constitution in 1789.[2]
 
The tra­jec­to­ries sketched by V‑Dem and by the Bright Line Watch experts are remark­ably con­sis­tent over time, though the BLW assess­ments are a bit more tempered (slightly higher than V‑Dem from the mid-19th century to the mid-20th, slightly lower since). Notably, both indices drop sharply in the last few years of the series. V‑Dem’s data end in 2017 but BLW experts perceive further decline in 2018 (from 76 in 2015 to 69 in 2017 to 64 now). We interpret this plunge as the response to the events of the 2016 U.S. pres­i­den­tial election and the Trump pres­i­den­cy.[3]

We next turn to a com­par­i­son between experts and the public. The next figure illus­trates changes in per­cep­tions of American democracy since 2015. In addition to pre­sent­ing ratings from our academic experts and a from rep­re­sen­ta­tive sample of Americans, it also breaks out the latter group into those who specif­i­cal­ly approve and dis­ap­prove of President Trump. 

Comparing the BLW expert sample with the public overall, we see that the experts rate American democracy higher than the public on average in each survey[4], but that both experts and the public perceive a continued erosion in demo­c­ra­t­ic quality over the past year. From September 2017 (the first survey for which we have both public and expert data) to July 2018, the average scores decline from 69 to 64 for the experts, and from 58 to 55 among the public. 

However, this overall decline also masks growing polar­iza­tion within the public between Trump approvers and dis­ap­provers. Trump approvers have rated U.S. democracy higher than dis­ap­provers in each of our public surveys, but the ratings pre­vi­ous­ly tended to move roughly in parallel. In the most recent survey, however, we see a sharp diver­gence. The democracy rating among Trump sup­port­ers increased from 58 to 63 while the rating among Trump opponents declined from 53 to 49, widening the gap from 5 to 14 per­cent­age points (its previous high was 9 per­cent­age points in January).

The two figures presented above paint a troubling picture. The over­ar­ch­ing pattern of progress achieved over the last two centuries is now reversing. The decline of 2016 has continued through­out the first eighteen months of the Trump pres­i­den­cy according to the assess­ments of both experts and the public (on average). We also observe increas­ing polar­iza­tion between Trump’s sup­port­ers and his opponents over the status of democracy in America.

 

Unpacking per­for­mance and polar­iza­tion: 27 demo­c­ra­t­ic principles

In addition to assessing the overall quality of democracy, our expert and public survey respon­dents rated per­for­mance on 27 distinct demo­c­ra­t­ic prin­ci­ples.[5] Our main metric of per­for­mance here is the per­cent­age of respon­dents who rated the United States as “fully” or “mostly” meeting a given demo­c­ra­t­ic standard, as opposed to the per­cent­age saying the United States only “partially meets” or “does not meet” it.[6]

The figure below shows the per­cent­age of Trump sup­port­ers and opponents who rate the U.S. as fully or mostly meeting the standard proposed by each statement of principle (e.g., “Government does not interfere with jour­nal­ists or news orga­ni­za­tions”). Statements are listed in descend­ing order of per­for­mance as assessed by those who approve of the president. The prin­ci­ples listed higher in the graph tend to be those on which opinions diverge the most between pro- and anti-Trump respon­dents — on whether equal rights are guar­an­teed to all citizens, on the effec­tive­ness of insti­tu­tion­al checks on executive authority, and on the fairness of elections. On these matters, those who dis­ap­prove of Trump rate American democracy poorly whereas the president’s sup­port­ers give our system high marks. 

The largest gap we observe is for the question asking whether U.S. elections are free from foreign influence — 57% of Trump sup­port­ers, but only 14% of opponents, say the U.S. fully or mostly meets the standard. These numbers did not differ sig­nif­i­cant­ly by whether respon­dents were surveyed before or after President Trump’s Helsinki summit with Vladimir Putin, which took place while the survey was in the field.

On other items, the divide between Trump sup­port­ers and opponents is less stark, in part because both groups see per­for­mance as lacking. Two prin­ci­ples that stand out as examples are that gov­ern­ment agencies do not punish political opponents (32% of sup­port­ers and 29% of opponents say the U.S. meets the standard “fully” or “mostly”) and that inves­ti­ga­tions into official mis­con­duct are not com­pro­mised (28% for sup­port­ers, 22% for opponents). Rather than signaling a har­mo­nious bipar­ti­san consensus, however, similar ratings on these prin­ci­ples might indicate that both sides see the other as respon­si­ble for under­min­ing demo­c­ra­t­ic practices. In crit­i­ciz­ing the FBI, Special Counsel Robert Mueller, and prominent intel­li­gence officials, President Trump claims he is under siege by a supposed “deep state.” By contrast, Trump critics might regard the president’s attacks as com­pro­mis­ing the inde­pen­dence of agencies and inves­ti­ga­tions. We discuss this issue further in the next section of this report.

As in prior surveys, both Trump sup­port­ers and opponents give low ratings to prin­ci­ples that reflect norms of civility in political behavior: rec­og­niz­ing the legit­i­ma­cy of political adver­saries, seeking com­pro­mise, and finding a common under­stand­ing of facts. Finally, both groups rate the use of public office for private gain as an area of poor demo­c­ra­t­ic performance. 

We have doc­u­ment­ed the divide in assess­ments of U.S. democracy between sup­port­ers and opponents of the president in previous reports.[7] Here, we drill deeper into the growing diver­gence by examining the extent to which it is reflected across our 27 prin­ci­ples. The next two figures show per­for­mance assess­ments, first by Trump sup­port­ers and then by opponents, on each item over our two most recent surveys (April and July 2018). The items are listed in declining order of each group’s ratings on the most recent survey, which are indicated by the darker markers in each graph. The note­wor­thy pattern is that the president’s sup­port­ers rate demo­c­ra­t­ic per­for­mance higher in July than in April on 23 of 27 items (four per­cent­age points higher on average). Among Trump opponents, by contrast, the pattern is com­plete­ly reversed, with declines on 25 of 27 items from April to July (by six per­cent­age points on average). As a result, overall public ratings are largely stable; see the figure in the appendix. 

The uni­for­mi­ty across items of the shifts within each group is as striking as the polar­iza­tion across groups. Those who dis­ap­prove of President Trump regard American democracy as declining on virtually all fronts, and steeply on many. Those who approve of the president regard demo­c­ra­t­ic standards as improving nearly across the board. The dis­junc­ture is more pro­nounced now than in previous surveys. Just six months ago, we reported on the changes in ratings on the same 27 prin­ci­ples between our September 2017 and January 2018 surveys. At that time, both groups shared similar per­cep­tions of demo­c­ra­t­ic decline. On 22 of the 27 prin­ci­ples, there was no sta­tis­ti­cal­ly dis­cernible dif­fer­ence between groups in the change over time. By contrast, we now see a sig­nif­i­cant­ly greater relative decrease in perceived per­for­mance among the president’s opponents on 14 of 27 items when comparing April and July 2018. 

In summary, President Trump’s sup­port­ers and his detrac­tors are expe­ri­enc­ing the same events during his time in the White House and drawing con­clu­sions about democracy that are increas­ing­ly opposed. 

 

Speech and policy: What’s (ab)normal, what’s important

Critics and sup­port­ers of Donald Trump alike highlight the degree to which the president is uncon­strained by con­ven­tion. To measure the impor­tance of President Trump’s norm vio­la­tions, our expert survey included an addi­tion­al battery of questions asking respon­dents to assess a series of recent political events on their impor­tance and on the degree to which they fit with normal political practice.[8] The list included the following 22 items:[9]

  • President Trump holding a face-to-face summit with North Korean leader Kim Jong-Un

  • President Trump describing members of the MS-13 gang as “animals”

  • The Department of Homeland Security separating children from parents who enter the United States illegally

  • President Trump criticizing Canadian Prime Minister Justin Trudeau as “dishonest and weak”

  • Congress passing tax legislation

  • President Trump pardoning Dinesh D’Souza for making illegal campaign contributions

  • President Trump repeatedly calling the press an “enemy of the people”

  • The Trump administration reversing Obama-era environmental and financial regulations

  • President Trump calling for suspending due process rights for suspected illegal immigrants

  • President Trump profiting from political fundraisers and events held by foreign dignitaries at hotels and resorts owned by his family business

  • President Trump calling for the Department of Justice to investigate Hillary Clinton

  • President Trump publicly criticizing the investigation led by special counsel Robert Mueller

  • President Trump nominating Brett Kavanaugh to replace Supreme Court Justice Anthony Kennedy

  • The United States withdrawing from the United Nations Human Rights Council

  • President Trump congratulating Vladimir Putin on his election victory

  • President Trump demanding greater defense spending from NATO allies

  • President Trump threatening American businesses such as Amazon and Harley-Davidson

  • The Trump Administration imposing tariffs on $34 billion in Chinese imports

  • The Trump administration imposing tariffs on steel and aluminum from Canada, Mexico, and the E.U.

  • President Trump publicly criticizing the F.B.I.

  • President Trump questioning the conclusions of U.S. intelligence about allegations of Russian election meddling during a press conference alongside Russian President Vladimir Putin.

  • President Trump publicly criticizing the leaders of the United Kingdom and Germany at a time when both are politically vulnerable at home.

Each respon­dent was asked to rate eight items (before the Helsinki summit) or nine items (afterward) that were randomly selected from the list above on two separate five-point scales according to the following instructions:

  • Normal or Abnormal, where 1 is business as usual for a presidential administration, like vetoing a bill or appointing a cabinet, and 5 is highly unusual for American democracy, like Iran-Contra or ordering newspapers to halt the publication of the Pentagon Papers.

  • Unimportant or Important, where 1 has limited or no consequences for the federal government, like the first family’s pets or the menu for a state dinner, and 5 creates a significant or lasting change, like the establishment of Social Security or the Voting Rights Act.

The following figure illus­trates the dis­tri­b­u­tion of state­ments across these two dimen­sions. The markers indicate the mean value of expert responses on each 5‑point scale. The items are more dispersed across the normal-versus-abnormal scale (on the hor­i­zon­tal axis) than on impor­tance (the vertical axis), where all 22 items were rated above the midpoint (“semi-important”) of the scale.[10] Two of the events that were rated as most important — the nom­i­na­tion of Brett Kavanaugh to the Supreme Court and Congress’s overhaul of the tax system — are also regarded as rel­a­tive­ly normal. To only a slightly lesser degree, the same is true for the Trump’s efforts to reverse Obama admin­is­tra­tion reg­u­la­tions on the envi­ron­ment and the financial sector. The con­se­quences of these policies are sub­stan­tial, but both are con­sis­tent with standard practice under recent admin­is­tra­tions. Experts also rated Trump’s demand that NATO allies increase their defense spending as less con­se­quen­tial but still normal. Presidents of both parties have pushed NATO partners in the same direction (albeit typically more discreetly).

 

Of the 22 events we selected, 18 are rated on the abnormal side of the scale (above the midpoint of “bor­der­line normal”). We start with those items rated on average between “bor­der­line normal” and “mod­er­ate­ly abnormal,” which include a number of events that range from rel­a­tive­ly high to rel­a­tive­ly low on the impor­tance scale. Our experts regarded Trump’s pardon of Dinesh D’Souza as only semi-important, notwith­stand­ing spec­u­la­tion that the move estab­lish­es a precedent for the arbitrary use of the pardon power for political allies in the future. They similarly dis­count­ed the impor­tance of Trump’s reference to MS-13 gang members as “animals” and his phone call to con­grat­u­late Russian President Vladimir Putin on victory in a tainted election despite sub­stan­tial outcries about both events at the time. And they regarded the United States’s with­draw­al in June from the United Nations Human Rights Commission as less than momentous, perhaps because of the body’s mixed record for influence and the checkered human rights records of many members. 

Other items in this range of abnor­mal­i­ty were regarded as more con­se­quen­tial. President Trump’s summit with North Korean leader Kim Jong Un, despite the lack of concrete policy agree­ments, falls in this category. So, too, do two more sub­stan­tive policy actions that Trump was able to execute uni­lat­er­al­ly — imposing tariffs on a range of Chinese goods, and on steel and aluminum imports from U.S. allies including Canada, Mexico, and the European Union.

On the right-hand side of the figure are the most dramatic depar­tures from normal politics — those rated as on average between “mostly abnormal” to “abnormal” by our experts. The most crowded sector of the figure is at the upper right where items rated high on both abnor­mal­i­ty and impor­tance are clustered. The highest combined rating here goes to President Trump’s state­ments in Helsinki, where he sided with Russian President Vladimir Putin and against American intel­li­gence agencies on whether Russia inter­fered with the U.S. election in 2016. 

Strikingly, most of the items that stand out as both important and abnormal, like the Putin press con­fer­ence, are not sub­stan­tive policy actions but rather verbal attacks by President Trump on political adver­saries. The president’s repeated char­ac­ter­i­za­tions of the press as an “enemy of the people” score high on both scales, as do his crit­i­cisms of the FBI, of the Mueller inves­ti­ga­tion, and of British Prime Minister, Theresa May, and German Chancellor, Angela Merkel. In a similar category would be his calls (so far empty) for criminal inves­ti­ga­tions into his former political opponent, Hillary Clinton, and to suspend due process for those suspected of entering the country illegally.

Slightly lower in impor­tance, but still rated as abnormal, are President Trump’s criticism of Canadian Premier Justin Trudeau as “dishonest and weak” and his threats (again, empty so far) to use reg­u­la­to­ry power to punish private sector actors such as Harley-Davidson for con­sid­er­ing moving pro­duc­tion overseas, or Amazon.com for the critical reporting by The Washington Post, which is owned by Amazon CEO Jeff Bezos.

The right-hand side of the figure is not entirely without sub­stan­tive policy content. Most important here is the sep­a­ra­tion of families entering the United States illegally at the southwest border. Separating thousands of children from their parents, many of whom have not been reunited, triggered wide­spread protests, legal chal­lenges, and even public rebuke from some nation­al­ly prominent Republicans before the Trump admin­is­tra­tion even­tu­al­ly walked back from the policy. Also in this sector is the ongoing practice by the president and his family of profiting from patronage at Trump prop­er­ties by political actors and foreign dig­ni­taries, which our experts as important and abnormal. 

These excep­tion­al cases aside, however, the events that our experts rate as most abnormal and important are over­whelm­ing­ly speech acts that are uncon­nect­ed to any formal pres­i­den­tial authority or public policy (as in prior Upshot surveys in February and May 2017). President Trump’s practice of airing crit­i­cisms and threats publicly and pre­emp­tive­ly con­tributes to the sense that these speech acts are not normal. But because they lack policy substance, one might rea­son­ably question the impor­tance our experts attribute to them. 

The case for their impor­tance depends on whether Trump’s speech is even­tu­al­ly trans­lat­ed into policy or influ­ences the behavior of other actors, including Congress, the federal bureau­cra­cy, and future pres­i­dents. In some cases, Trump’s extreme rhetoric might expand the range of what policy options are con­sid­ered rea­son­able or deter political oppo­si­tion. Of most direct concern to Bright Line Watch, however, are speech acts that encourage the violation of essential demo­c­ra­t­ic practices. When the president calls for the criminal inves­ti­ga­tion of a political opponent, sus­pend­ing an inves­ti­ga­tion of an ally (or of the president himself), or denying to a whole category of people due process explic­it­ly granted in the Constitution, the impli­ca­tions are ominous even if policy changes do not follow imme­di­ate­ly. If pres­i­den­tial speech in these areas even­tu­al­ly changes how other political actors under­stand the scope of con­sti­tu­tion­al pro­tec­tions or the bound­aries around the political use of gov­ern­ment authority, then President Trump’s trans­gres­sive speech could represent an exis­ten­tial threat to American democracy. That is why, we suspect, experts regard these speech acts as both highly important and highly abnormal. 

 

Appendix:  Changes in ratings of U.S. demo­c­ra­t­ic per­for­mance, public sample

Appendix:  Ratings of ratings of U.S. demo­c­ra­t­ic per­for­mance, expert sample

Appendix: Survey method and data

 

Bright Line Watch surveys on the state of America’s democracy, July 2018

From July 7–23, 2018, Bright Line Watch conducted its sixth survey on the state of democracy in the United States. We conducted previous surveys in February (Wave 1), May (Wave 2), and September (Wave 3) of 2017, and in January (Wave 4) and April (Wave 5) of 2018. Waves 1 and 2 targeted expert respon­dents only. Waves 3–6 have paired the expert survey with a rep­re­sen­ta­tive survey of the American public. Details on the Wave 6 survey are provided below:

  • Expert: On July 7, we sent email invitations to 9,195 political science faculty at universities in the United States. By July 23, after two reminder emails, we had complete responses from 679 (a response rate of 7.4%).

  • Public: YouGov fielded the public survey from July 11–18, producing 2,000 complete responses.

Both the expert and public samples in Wave 6 responded to a battery of questions about demo­c­ra­t­ic per­for­mance in the United States. Afterward, they were asked to evaluate the quality of American democracy overall on a 100-point scale. Expert respon­dents were then asked to respond to a second battery in which they were presented with a series of state­ments about current political events and asked to rate them on normalcy and impor­tance. The specifics of each battery are described below. The data from both the expert and public surveys are available here. All analyses of the public data from YouGov incor­po­rate survey weights.

 

Performance battery

The foun­da­tion of Bright Line Watch’s surveys is a list of 27 state­ments express­ing a range of demo­c­ra­t­ic prin­ci­ples. Democracy is a mul­ti­di­men­sion­al concept. Our goal is to provide a detailed set of measures of demo­c­ra­t­ic values and of the quality of American democracy. We are also inter­est­ed in the resilience of democracy and the nature of potential threats it faces. Based on the expe­ri­ences of other countries that have expe­ri­enced demo­c­ra­t­ic setbacks, we recognize that demo­c­ra­t­ic erosion is not nec­es­sar­i­ly an across-the-board phe­nom­e­non. Some facets of democracy may be under­mined first while others remain intact, at least initially. The range of prin­ci­ples that we measure allows us to focus attention on variation in specific insti­tu­tions and practices that, in com­bi­na­tion, shape the overall per­for­mance of our democracy.

Bright Line Watch’s Wave 1 survey included 19 state­ments of demo­c­ra­t­ic prin­ci­ples. Based on feedback from respon­dents and con­sul­ta­tion with col­leagues, we expanded that list to 29 state­ments in Wave 2. We then reduced that set to what we intend to be a stable set of 27 state­ments for the Wave 3 through Wave 6 surveys. 17 of those 27 state­ments were included in Wave 1, and all 27 were included in Wave 2.

The full set of state­ments is presented below and grouped the­mat­i­cal­ly for clarity. In the surveys, the prin­ci­ples were not cat­e­go­rized or labeled. Each respon­dent was shown a randomly selected subset of nine state­ments and asked to first rate the impor­tance of those state­ments and then rate the per­for­mance of the United States on those dimensions.

 

27 state­ments of demo­c­ra­t­ic principles

Elections

  • Elections are conducted, ballots counted, and winners determined without pervasive fraud or manipulation

  • Citizens have access to information about candidates that is relevant to how they would govern

  • The geographic boundaries of electoral districts do not systematically advantage any particular political party

  • Information about the sources of campaign funding is available to the public

  • Public policy is not determined by large campaign contributions

  • Elections are free from foreign influence

Voting

  • All adult citizens have equal opportunity to vote

  • All votes have equal impact on election outcomes

  • Voter participation in elections is generally high

Rights

  • All adult citizens enjoy the same legal and political rights

  • Parties and candidates are not barred due to their political beliefs and ideologies

  • Government protects individuals’ right to engage in unpopular speech or expression

  • Government protects individuals’ right to engage in peaceful protest

  • Citizens can make their opinions heard in open debate about policies that are under consideration

Protections

  • Government does not interfere with journalists or news organizations

  • Government effectively prevents private actors from engaging in politically-motivated violence or intimidation

  • Government agencies are not used to monitor, attack, or punish political opponents

Accountability

  • Government officials are legally sanctioned for misconduct

  • Government officials do not use public office for private gain

  • Law enforcement investigations of public officials or their associates are free from political influence or interference

Institutions

  • Executive authority cannot be expanded beyond constitutional limits

  • The legislature is able to effectively limit executive power

  • The judiciary is able to effectively limit executive power

  • The elected branches respect judicial independence

Discourse

  • Even when there are disagreements about ideology or policy, political leaders generally share a common understanding of relevant facts

  • Elected officials seek compromise with political opponents

  • Political competition occurs without criticism of opponents’ loyalty or patriotism

 

The Wave 6 survey consisted of two main parts. In the first, each respon­dent was asked, “How well do the following state­ments describe the United States as of today?” Each respon­dent was then presented with 14 state­ments of principle, randomly drawn from the set above, and offered the following response options:

  • The U.S. does not meet this standard

  • The U.S. partly meets this standard

  • The U.S. mostly meets this standard

  • The U.S. fully meets this standard

  • Not sure

The order in which state­ments were presented in each battery was ran­dom­ized for each respon­dent so there should be no priming or ordering effects in how they were assessed.

After com­plet­ing the battery on U.S. per­for­mance, we asked respon­dents to rate the overall quality of democracy in the United States today on a scale from 0 to 100, where 0 is least demo­c­ra­t­ic and 100 is most democratic.

Notes

[1]  Bright Line Watch col­lab­o­rat­ed with The Upshot (where BLW’s Brendan Nyhan is a con­trib­u­tor) in designing the original questions and con­struct­ing the sample of respon­dents and consulted with The Upshot before fielding the current battery. Special thanks to The Upshot’s Claire Cain Miller.

[2] V‑Dem is a scholarly con­sor­tium that evaluates the quality of democracy around the world and over time using con­sis­tent methods and standards to aggregate the opinions of experts in 170 countries.

[3] For more dis­cus­sion of the ret­ro­spec­tive assess­ments and how they cor­re­spond to devel­op­ments in American political devel­op­ment, see the report from the BLW May 2017 survey.

[4] We suspect that the overall higher ratings among experts are driven primarily by their greater famil­iar­i­ty with politics in countries outside the United States, where democracy and basic rights and liberties are often com­pro­mised. For more dis­cus­sion, see the report from the BLW September 2017 survey.

[5] Each respon­dent rated 14 prin­ci­ples randomly selected from the full set of 27. The complete list of state­ments is presented in an appendix to this report. In the survey, respon­dents rated per­for­mance on the indi­vid­ual prin­ci­ples prior to being asked to provide an overall rating on the 100-point scale.

[6] We omit a dis­cus­sion of changes in expert ratings on the 27 prin­ci­ples in this report due to their general stability. A figure showing these changes is in an appendix.

[7] We direct inter­est­ed readers in par­tic­u­lar to this summary of the first year of BLW surveys.

[8] As noted above, the design of this survey is modeled on previous surveys conducted by The Upshot at the New York Times.

[9] The initial list included just the first 20 state­ments. Shortly after the survey went into the field, President Trump traveled to Europe to attend a meeting of chief exec­u­tives from NATO member countries in Brussels, visit the United Kingdom, and take part in a summit with Russian President Vladimir Putin in Helsinki, Finland. The president made a number of con­tro­ver­sial state­ments during this trip. As a result, we added the last two state­ments to the battery while the survey was in the field.

[10] The figure therefore truncates the unim­por­tant-to important scale at its midpoint to allow greater res­o­lu­tion in the upper section of the scale.