Corpus strategies and pragmatist complexity

Francis Chateauraynaud and Josquin Debaz (GSPR – EHESS, Paris)

Read the second part of this text Pragmatic sociology and the digital worlds: towards an alternative laboratory

Big Data presents a number of opportunities for social scientists and humanities scholars, not least of which are massive quantities of very rich social, cultural, economic, political and historical data. It also poses a number of challenges, including a skills deficit for analysing and making sense of such data, and the creation of an epistemological approach that enables post-positivist forms of computational social science. (Kitchin, 2014).

Defining the best strategies to build relevant architectures of corpus

According to Rob Kitchin, taking seriously the rise of Big Data is

coping with abundance, exhaustiveness and variety, timeliness and dynamism, messiness and uncertainty, high relationality, and the fact that much of what is generated has no specific question in mind or is a by-product of another activity. (Kitchin, 2014)

The conjunction of high-powered computation and new analytical techniques has changed the landscape. It opens a new era for social scientists able to mobilize new computer-based techniques, in order to automatically mine data, detect patterns, build predictive models or stimulate new social theories (Schroeder, 2018).

Considering that “Big Data is a disruptive innovation”, without fully eliminating small data studies, Kitchin examines the epistemological stakes, by identifying two potential paths that have divergent epistemologies:

empiricism,wherein the data can speak for themselves free of theory, and data-driven science that radically modifies the existing scientific method by blending aspects of abduction, induction and deduction (Kitchin, 2014).

In his conclusion, he adds:

Given the weaknesses in the empiricist arguments it seems likely that the data-driven approach will eventually win out and over time, as Big Data becomes more common and new data analytics are advanced, will present a strong challenge to the established knowledge-driven scientific method.(Kitchin, 2014)

What strategy can researchers adopt in the humanities and social sciences? Given the diversity of theoretical and philosophical traditions, the reaction to the “era of data” will be obviously very different, depending of the types of data, of the need for contextualization, of timelines and of research questions. Our pragmatist approach of corpus analysis leads to develop a critical reading of the arguments based on a general shift in social science epistemology due to the rise of Big Data (For a very useful and relevant discussion see Jean-Philippe Cointet et Sylvain Parasie? 2018). As we have seen, we cannot escape interpretive approach to narratives and argumentations in order to make sense through textual or discursive data. Then we need a pluralistic model of building textual or discursive series,producing intellectual objects relevant for understanding the transformation of actors and arguments over time.

During the last years, by looking back on digital practices, we have coined six forms of strategies used to design and validate a relevant corpus or a constellation of corpora:

• The first type of corpus is made in order to confirm or to create an epistemic authority (like the Computational Analysis of Present-Day American English). The main issue here is the crucial stability of the documents and categories, stability which relies on the internal coherence that experts of the field maintain;
• A second pathway to generate a corpus is a formal procedure based on keywords or metadata, and controlled database or data flow (usually they are meant to be repeated and often automated). This process allows a procedural delegation and the possibility of comparisons, as long as sources remain stable;
• The third strategy emerges from a collaborative accumulation, whose maintenance needs a dedicated governance – as in Wikipedia experience, a collective of contributors feeds, analyses, discusses and maintains the corpus;
• In front of the three first types, we find a fourth strategy in which the corpus is composed by the aggregation of various smaller corpora. In real practice, generally for technical reasons, completeness and congruence of data is partially suspended in order to associate corpora derived from different methods. The validation of the final corpus has to be built through some heuristic model of testing;
• A fifth approach is characterized by a multiplicity of corpora, which data are too heterogeneous to be made compatible with the same tools. In this case, the key element lays on the formal links that enable a meaningful circulation between corpora;
• Lastly, an important form of corpus depends on a severe selection of data, by eliminating structural redundancy, giving an informational-loss free minimum size corpus. A technique used in Prospéro studies is to verify if some relevant references in the texts are included in the selective corpus – called Okham’s corpus in our epistemic community.

These different types are far from being exclusive. A interesting procedure can be based on a transformative process by which epistemic authority, automatic method and collaborative discussion will be confronted during the whole process of research in order to produce the best cognitive pathway. A good example of application has been the case of the constellation of corpus made during the long run study of GMOs controversies (Chateauraynaud, 2010).

This figure shows the recent state of the French selected corpus concerning GMOs in Europe. Composed of more than 12.000 texts, this collection of texts gives a strong basis for investigating new events in the field of biotechnologies in agriculture. Depending on the research questions or on the collective discussions, we can alternatively use an aggregative corpus, an authoritative corpus or a selected corpus.

Towards a pragmatics of transformations working on multi-scales and non-linear processes

Our corpus collection is full of reports, petitions, debates,opinions, scientific articles, political decisions, media coverage,etc. The text databases are associated with analytical grids adapted to the diverse arenas through which knowledge and politics are brought together, producing consequences on institutions, especially on regulatory agencies. Nowadays, official expert systems are often suspected of tending toward a form of “epistocracy” and citizen concerns are often limited to media expressions issues¹. To overcome the opposition between “sound science” and “science engaged in civic or social epistemology”, we have distinguish three regimes of evidence:

• An axiomatic regime (formal and syntactic way of reasoning, producing autonomous system of proof);
• a conventional regime (process of agreement on natural and social categories connected by forms of equivalence, and generating social ontologies);
• a phenomenological regime (cross-checking clues, signs and facts in sensitive and material world).

By following long term controversies we can describe how actors engage these three logics, all necessary to produce strong evidence and to find an agreement between players defending heterogeneous or opposite interpretations and normative visions. In order to address complexity and uncertainty, non-linear processes and emerging issues, long run trends and ruptures, we need a form of pragmatics of complexity². This is the reason why research in socio-informatics is so often oriented to collaborative investigations of multi-scale controversies. By re-assembling quantitative data analysis,argumentative theories and qualitative inquiries, including field work and ethnography, we seek to build a new methodological alliance for interpreting the dynamics of public issues connecting science and politics. At this stage of our reasoning, there is a resonance with the pragmatist claim about the importance of abductive reasoning:

Instead of using Peirce’s theory of inference as a way to legitimize grounded theory, we thus construct abductive analysis from a grounded theory foundation to foster theoretical innovation. Abductive analysis constitutes a qualitative data analysis approach aimed at theory construction. This approach rests on the cultivation of anomalous and surprising empirical findings against a background of multiple existing sociological theories and through systematic methodological analysis. As such, it requires a fundamental rethinking of core ideas associated with grounded theory, specifically the role of existing theories in qualitative data analysis and the relationship between methodology and theory generation.” (Timmermans and Tavory, 2012)

One consequence of this exploratory pathway, operating between quantitative data processing and interpretive analysis, is the emergence of a new epistemic space for the formalization of argumentation and reasoning. In front of long run processes, the revision procedures are key elements to address (see before our comment on Gärdenfors and Makinson). Avoiding a reductionist understanding of argumentative activity, as it is the case when argumentation is considered only under the norm of classic logics, were place the canonical form expressed by the general clause “if P, then Q,” by more complex clauses. Then, we have to deal with formula deriving from clauses like: “if P, then M [temporal modalisation and mode of social agreement] Q”. For instance, we must take into account phrases like: “if P, then maybe,after a probable agreement of players, there will soon be Q.” As a well-formed and realistic example: “if the climate change process and the sixth extinction of species are not curbed within a decade, humanity will disappear from the Earth”…

Forms of logics and pragmatic sociologies: from modus ponens to abductive reasoning

Classical mode of reasoning Modus Ponens / Modus Tollens	Monotonic Logics
P implies Q; P is asserted to be true, so therefore Q must be true.	Foundation of classical rationality
Pragmatics of Justification (Pragmatics according to Boltanski and Thévenot, 2006)	Putting Rationality in context and connecting social situations to universal principles of foundation
S (situation) congruent to R (Regime of Justification), then S implies R – if not true, then S implies D (Dispute)	The plurality of Principles or Regimes of action and/or justification explains the numerous disputes in social life
Pragmatist sociology of transformation Events, actions and argumentations are transformed through interpretive activities, producing a new series of actions, argumentations and decisions, with a huge uncertainty on practical consequences	The plurality of interpretations creates both constant reevaluations of norms and rules, and a creative/collective activity with emerging actors, new ways of arguing and innovative devices or tools.
In a situation S, E (event) or A (action) seized by I (interpretative activity) produces A (argumentation) from which proceeds the evaluation or valuation (V) of available rules (R0… Rn) but, at the same time, of known and unknown consequences (C) which evaluation or valuation (V) retroacts not only on the situation S but on a series of future situations (F)	Non-monotonic or complex logics with emerging and retroactive consequences (non-linear process and feedback)

In complex issues, events narratives, actions and argumentations are transformed through interpretive activities, producing a new series of unforeseen actions, argumentations and decisions, uncertainty being reinforced by unpredictable developments of “interacting milieus” (Chateauraynaud and Debaz, 2017). Along public controversies, the plurality of interpretations creates both constant re-evaluations of norms and rules, and a creative collective activity, producing emerging actors and new devices (dispositifs). The table above helps to figure out the huge differences between a pragmatics of complexity and a sociology of justification based on classical logics.

General Conclusion

In this paper distributed in 3 posts, we have exposed three key drivers for the renewing our social science interpretations in front of “Big Data” wave. First, keeping a strong grasp on argumentative and narrative activities by actors, allows social scientists to follow the production of meaning in socio-historical contexts. The second dimension concerns the making of meaningful collections of different type of corpus by assuming a maximal reflexivity on data and metadata involved in social science investigations. Thirdly, we showed that a strong modelling of complex social systems is to be explicitly developed in order to avoid naïve views of the common idea of complexity (Chavalarias, 2009). Using digital tools designed for analysing social processes creates an opportunity to show how a pragmatist sociology gets some epistemic advantage by using non-monotonic or complex logic in order to interpret emerging and retroactive consequences conveyed by non-linear processes and surprising feedback. In front of this challenge, Big Data is maybe less a solution than a problem. Some scientists, joined by corporations and governments believe in Big Data horizon and call for a mobilization of social scientists. For us, it is a new issue to follow, by returning the quantitative dragoon produced by flows of texts on the web in a qualitative exploration of arguments in a pragmatist laboratory!

---

General References

Abbott (2001) On the concept of turning point. In: Time matters: on theory and method. Chicago: University of Chicago Press: 245-248.

Angenot M (2008) Dialogue de sourds: traité de rhétorique antilogique. Essai. Paris: Mille et une nuits.

Beck U. (2008) World at Risk. Cambridge (UK): Polity Press.

Brennan J (2016) Can epistocracy, or knowledge-based voting, fix democracy?” (Opinion). Los Angeles Times. Aug.28^th.

Chateauraynaud F (2010) Marloweb.ogm:(trans)genèse d’une promesse méthodologique. Socio-informatique et argumentation. September 19. https://socioargu.hypotheses.org/1421.

Chateauraynaud F (ed.) (2014) Une pragmatique des alertes et des controverses en appui à l’évaluation publique des risques. Paris: ANSES (December).

Chateauraynaud F and Chavalarias D (2018) L’analyse des grands corpus évolutifs et la sociologie pragmatique des controverses. Croiser les méthodes face aux transformations des mondes numériques. Sociologie et Sociétés. Forthcoming.

Chateauraynaud F and Debaz J (2017) Aux bords de l’irréversible: sociologie pragmatique des transformations. Paris: Pétra.

Chateauraynaud F and Doury M (2010) The Collective making of temporal Aspects in Public Debates. In: Proceedings of the 7th Conference of the International Society for the Study of Argumentation. Amsterdam.

Chavalarias D (2009) Désir mimétique et imitation rationnelle. Vers un individualismeméthodologique complexe. In: Anspach M Rand Centre culturel international de Cerisy-la-Salle (eds). Jean-Pierre Dupuy, dans l’œil du cyclone : colloque de Cerisy. Paris:Carnets nord.

Cointet D. et Parasie S. (2018) « Ce que le big data fait à l’analyse sociologique des textes. Un panorama critique des recherches contemporaines », Revue française de sociologie, 2018/3 (Vol. 59), p.533-557

Foucault M (1971) Orders of discourse. Social Science Information 10(2): 7–30. DOI: 10.1177/053901847101000201.

Foucault M (1982) The archaeology of knowledge. New York, NY: Pantheon Books.

Funtowicz SO and Ravetz JR (1993) Science for the post-normal age. Futures 25(7): 739–755. DOI: 10.1016/0016-3287(93)90022-L.

Foucault M and Kriegel B (eds) (1975) I, Pierre Rivière, having slaughtered my mother, my sister, and my brother: a case of parricide in the 19th century. 1st American ed. New York: Pantheon Books.

Gärdenfors P and Makinson D (1988). Revisions of knowledge systems using epistemic entrenchment. In: Proceedings of the Second Conference on Theoretical Aspect of Reasoning About Knowledge. San Francisco: Morgan Kaufmann Publishers Inc. 83–96.

Keller R (2012) Doing discourse research: an introduction for social scientists. Los Angeles: SAGE.

Keller R (2018) The Sociology of Knowledge Approach to Discourse. An Introduction. To be published.

Kitchin R (2014) Big Data, new epistemologies and paradigm shifts. Big Data & Society 1(1): 205395171452848. DOI:10.1177/2053951714528481.

Landemore H (2017) Democratic Reason – Politics, Collective Intelligence, and the Rule of the Many. Princeton:Princeton University Press.

Lyotard J-F (1983) Le différend. Collection ‘Critique’. Paris: Editions de Minuit.

Madiot B. (2013). Analysing the French Abstracts from the International Conferences of Social Representations. Papers on Social Representations. 22: 10.1-10.23. http://psr.iscte-iul.pt/index.php/PSR/article/view/325

Perelman C and Olbrechts-Tyteca L (1969) The new Rhetoric: a treatise on Argumentation. Notre Dame: University of Notre Dame Press.

Radermacher W (2016) Systems, resilience, risks and accounting. In:Report of the EEA Scientific Committee Seminar on emerging Systemic Risks. Copenhagen, 24 February.

Roe E (2013) Making the most of mess: reliability and policy in today’s management challenges. Durham ; London: Duke University Press.

Ripple WJ, Wolf C, Newsome TM, et al. (2017) World Scientists’ Warning to Humanity: A Second Notice. BioScience 67(12): 1026–1028. DOI: 10.1093/biosci/bix125.

Schroeder R (2018) Social Theory after the Internet. London: UCLPress.

Smith S (2017) Using speech acts formulas to describe the narrative structure of affaires. Socio-informatique et argumentation. April, 27.

Timmermans S and Tavory I (2012) Theory Construction in QualitativeResearch: From Grounded Theory to Abductive Analysis. Sociological Theory 30(3): 167–186. DOI: 10.1177/0735275112457914.

Tindale CW (1999) Acts of arguing: a rhetorical model of argument. SUNY series in logic and language. Albany, N.Y: State University of New York Press.

Tsing Anna and Pollma Elizabeth (2005). Global Futures: The Game. In: Rosenberg D and Harding SF (eds). Histories of the Future. Durham: Duke University Press.

Walton DN (2008) Informal logic: a pragmatic approach. 2nd ed. Cambridge ; New York: Cambridge University Press.

1. See (Landemore, 2017). A debate in the US was opened, reproducing a very old opposition between experts and lay people. See for example (Brennan, 2016). [↩]
2. This sociological approach of complexity shares some elements with the « post-normal science » – borrowing the claim developed in early 1990s by (Funtowicz and Ravetz, 1993). [↩]

---

---