Studying and practising justifying and accounting

Contextualisation as a research technique for staging controversies over field boundaries

Simon Smith,

Marie Curie Research Fellow, Institute for Sociology, Slovak Academy of Sciences.

Simon Smith has begun a collaboration with Prospéro’s team and network of developers, in order to adapt procedures and grids to a new kind of materials. This text is the starting point of a series of contributions and discussions, which will go on next year during the seminar on socio-informatics held at EHESS in Paris.

Introduction

If understanding what kind of case you’ve got (typical, critical, extreme, paradigmatic etc.) can emerge during the course of case study research (Flyvbjerg 2006) – and probably has to for fluid organisational settings and emerging fields – can you turn this from a liability to an asset? Flyvbjerg thought you could do so by cultivating a reactive, continual readiness to reinterpret. I argue that you can also in another, more proactive sense: by employing contextualisation as a research technique.

A year into my research project on media-hosted eParticipation in Slovakia, I re-read my project proposal.

In it I wrote that I wanted to:

  • combine “observation of the online environment, newsroom and other workplace-based observation with an institutional field analysis of a specific national context” and thereby “map the key dimensions of an emerging field”
  • combine different levels of analysis and thus “place questions connected to the future of journalism in the broader context of the evolution of democratic regimes and the changing modes of political participation which they sustain, showing how the constraints on professional practice are refracted through the structure of the field”

The initial analytical framework was to “take account of how this field is embedded in a national media system and how the case study organisation relates to the field as a whole” in the attempt to analyse “the actions and interactions of individual actors … against the backdrop of this institutional context”. Similarly, the knowledge exchange framework I proposed to use in action research elements of the project assumed I would be able to assess the types of knowledge that would be of most use to partners based on a “consideration of the context” in which people work.

The emphasised phrases indicate how I treated the context more or less as a given (and assumed it is delimited by a set of institutions embedded principally at the level of the nation state). It is something to be taken into account when designing the research, interpreting the results and translating the findings into messages that can be useful to organisational actors. I did not anticipate that it was going to especially problematic to understand and describe (indeed the proposal stressed my own prior familiarity with the social, political and cultural context of Slovakia) and I did not view the research as an active intervention in context – as a contextualising force.

A few months in I had already realised the need for a more nuanced research design, iteratively combining bottom-up and top-down views of digital social worlds as I became familiar with the worlds of the actors who are reflexively engaged in the interpretation of closely overlapping knowledge corpuses to mine. I am now tempted to suggest that the sequence of steps that will enable me to “map the key dimensions of an emerging field” actually has to be reversed.

A field study?

Unlike most work in the sociology of arguments and critique, where corpuses are built by following issues chronologically and transversally,  I am studying a single organisation and treating it as the relevant field (or sub-field) of activity. Field studies normally have an overall dynamic that moves from the general to the particular, beginning by mapping the field as a whole and then homing in on particular regions, positions and actors within it. In a fluid and dynamic field like contemporary mass media organisations, where professional identities and credentials are undergoing radical mutations, it’s difficult to proceed in this order, since the boundaries of the field are unclear.

7 Skills of a Community Director -- The New Community Manager via @SocialOgilvy

7 Skills of a Community Director — The New Community Manager via @SocialOgilvy

Social journalists are typically said to need the same sort of ‘relationship skills’ that are demanded of candidates for almost any knowledge-related work. Online community managers are ascribed ‘transversal competences’. The necessary skill set may be evolving, as social media specialists Ogilvy claim, but it is fanciful to speak of an emerging profession because the skills bundled together are so diverse and unspecific. Indeed even a thorough empirical study like Estienne (2007) could not pin down a set of ‘core competences’ associated with online journalism. Mapping a field seemingly in a state of permanent renewal is provisional and indissociable from claims and challenges put forward by actors engaged in positional wars and boundary disputes. And hence what seem like relevant contextual factors at the start can turn out to be irrelevant, at least from the point of view of your research subjects/partners.

Unproductive research contextualisations

Let me give a few examples of how this affected my approach:

  • One way in which I planned to ‘map the context’ of my study was by interviewing officers of the national journalists’ trade union about ‘policy’ and legal questions in relation to journalism and eParticipation. But I quickly learned that the union is viewed with disparagement in the newspaper I’m studying. It’s not a source of authoritative rules or ‘journalistic consecration’: in fact it’s barely a reference in the symbolic field of action my respondents manoeuvre in. For authoritative pronouncements about how journalism should be practised they turn instead to foreign media considered to be paragons (notably the NY Times and the Guardian with respect to eParticipation).
  • the boundaries between professional jurisdictions within the organisation are extremely opaque and fluid. There are people who call themselves ‘reporters’ even though they work for the online division and do little original writing; there are people who have moved back and forth between reporting and editing positions; there are people occupying technical positions who nevertheless write as many articles as some full-time journalists, often about subjects that have little relation to their technical expertise; there are freelancers who combine journalism with non-journalistic activities. Given this chaotic mix of possible trajectories and combinations, the context in which eParticipation is performed varies greatly from person to person, since it is inserted within a different ‘bundle of tasks’ (Abbott 1988), and that affects the priority that is given to it and prestige attributed to it.
  • eParticipation is a practice that doesn’t have any formal scripts prescribing or controlling how it should be performed within the organisation. Instead its performance seems to draw entirely on tacit knowledge. This is not because it is unimportant – the quality of its readers’ blogs and online discussion, and the number of page visits to these domains, are the source of considerable pride for the organisation. Despite this, there is no formalised job description for tasks like online discussion administration. It doesn’t feature in the official organisational structure even though it is a time-consuming task that occupies a substantial part of the remit of half a dozen editors. It isn’t mentioned in the website pen portraits of these journalists or in any of their online public profiles (Linkedin, FB, Twitter, etc.). And it doesn’t have any documentary artefacts: there are no instruction manuals about how to do it or how to learn how to do it, and it isn’t monitored and assessed using work logs or databases. My attempts to treat eParticipation as part of both the journalistic and the organisational field keep running into this problem of non-codification – of the invisibility of any ostensive aspects of the routine (Pentland & Feldman 2005). It often seems as if I am trying to study a routine that exists only in its performative aspect and a professional identity – that of the eParticipation journalist – that no one wants to claim.

Rethinking contextualisation

Context is usually considered to be the social, economic, political or historical conditions which you need to ‘insert’ your findings or your data in in order to make proper sense of them, and to avoid misinterpretation. It is a crucial task in case study research because if we want our findings to be interpretable outside the given research setting, we need to give other readers ‘instructions’ about how (and how not) to read the results from their position, usually with a different context in mind. The trouble is twofold.

Firstly, in action research, context can become a means of explaining and justifying ‘why we don’t act as we should’ by invoking a series of organisational or institutional ‘contexts’ that allegedly constrain action. As I discovered when evaluating a collaborative research project, context fuels the formulation of ‘inverted reality tests’ that work against the collective reflexivity that action research requires:

[W]hereas the evaluation report referred to processual dimensions of organizing collaboration derived either from generic theoretical models or from the logic model, the research team indexed their explanations of the difficulties of collaboration to wider contextual factors such as the organizational instability of the local healthcare landscape and the management structure of the wider programme the research formed a part of. They normalized the emerging account of reality by recontextualizing it… [the] practical effect [of which] was to detach discussion at the workshop [organised to facilitate organisational learning] from the objects provided by the evaluators and reattach it to objects that were not present, and could not easily be made present at that moment in time, such as the institutions that shape academic research and government policy initiatives … With this ‘multiplication of measures’ (Périlleux, 2005), neither the planned nor the alternative reality tests were put into practice. (Smith, Ward & Kabele 2014, emphasis added)

So it isn’t just that context explains the divergence between how routines are performed and the rules or other abstract descriptions of how they are supposed to be performed (Pentland & Feldman 2005), it is that recontextualisation is a strategy participants use in the power disputes around the configuration of organisational routines.

Secondly, the very act of isolating certain objects for study and either identifying sources of data about them or inventing ways of generating data about them (what you might call your opening study choices) produce their context by demarcating a boundary between what’s in and what’s outside your study. Context is outside (though not everything outside is context). It denotes phenomena that have relevance for the object of study, but which you don’t study directly. You use them to frame your interpretations. It should be clear, therefore, that a different opening study choice could have defined the context differently by broadening or narrowing the actual object of study. Following this approach you are therefore very vulnerable to the validity of your initial boundary definitions.

The way round the first source of trouble could be to equalise the status and harmonise the temporal sequence of indigenous and ‘academic’ acts of contextualisation – to fully accept that you as a researcher are a player in the power games around organisational routines. The way round the second source of trouble could be to define context pragmatically (Chateauraynaud 2003: 193-4), exploring each data source as hypertext (an object that literally contains its own context in the form of all the references it explicitly or implicitly invokes), metaphorically asking each text who or what it ‘talks with’ and pragmatically delimiting the study by the limits of this network of interconnecting texts.

My revised approach to contextualisation can thus be defined in terms of two key propositions:

1. following the pragmatic sociological tradition, it’s possible to read texts as hypertextual artefacts that contain their own context

But I need to go further, since, as explained, I’m exploring a field that doesn’t generate many self-descriptive texts. Hence a second proposition:

2. (action) research co-produces texts that produce their own context

Viewing context in this way is particularly useful when studying emerging phenomena or uncodified organisational tasks and roles.

Here are two brief examples of how I’ve been attempting to put these propositions into practice.

Reading texts in a justificatory register

When observing online discussion administration being performed, there were no clearly marked contextual clues about how to interpret what was going on, the decisions that were being made, the factors that were being taken into consideration and the relation of the activity itself to the other tasks that make up ‘being a journalist’ (for the individual) or ‘running a newspaper’ (for the organisation). What I envisaged doing was looking at the standards, regulations and rules within which the activity takes place. Job descriptions, organisational procedures, training manuals, legal codes and the like would be my basic contextual elucidators. But none of these exist. Discussion administration is a social practice without any codified scripts in the case study organisation.

What I could do, on the other hand, was use research situations to produce textual artefacts that invoked an equivalent context. Instead of silently observing, I prompted administrators to justify each decision that they took, recording the qualifying words and phrases that they used. The result of this exercise was a series of texts in the form of lists of adjectival words and phrases. Adjectives attribute qualities to entities and thus singularise them by referring to a local ‘context’ within which the attributions apply, or by defining legitimate and illegitimate comparisons with other contexts (Chateauraynaud 2003: 231). They referred me to the discursive registers that mattered to those individuals at that particular time and place. They invoked a series of standards, norms and categories that oriented their judgement, including organisational policies and values as well as internalised, socialised thresholds of tolerance. This enabled quite a detailed recapitulation of tacit and embodied skills deployed by administrators, referring to (con)texts that do not formally script their work but are made present in its daily performance. Some refer very specifically to the organisational context while others enact a wider context ‘felt’ relevant by the administrator less as an organisational member than as a member of civil society with a sense of ‘decency’ or ‘normality’. Figure 1 shows some examples.

The administrator’s vocabulary of evaluation

The administrator’s vocabulary of evaluation

How to read figure 1

Black: justifications restricted to the semantic world of the discussion thread and the discussion community

Blue: justifications invoking organisational context (quoting keywords from the rules banning vulgarity, spam, personal attacks and insults; using available statistics; referring to the newspaper’s position and interests)

Red: justifications invoking a wider civic and political context that the administrator represents (or even embodies), typically relying on tacit knowledge and a sense of what is ‘normal’ or valuable

In subsequent analysis I added a second contextual layer by attaching their spontaneous enunciations to a theoretically-derived analytical framework, and then played them back to the group of administrators. This way we could, collectively, further contextualise the task of discussion administration by making explicit the specifically organisational or professional factors that constrain the performance of the task. Contextualisation became a process of testing the validity and viability of an emerging artefact that sought to describe the ostensive aspects of a routine. Effectively it went through three phases: externalising the tacit knowledge drawn on by administrators; linking to theoretical categories of discourse; invoking organisational constraints and professional norms. In the process, some of the earlier contextualisations were challenged and either refined or eliminated, to leave a textual artefact equipped with sufficient relevant contextual ‘hyperlinks’ to make it acceptable and useable.

Reading texts in an accounting register

When organisational spokespeople sum up this newspaper’s attitude to eParticipation they tend to stress the high importance of ‘user-generated content’ to their business model as well as a democratic mission to encourage and support civic and political engagement and free debate. When employees are asked to account for the organisation as a place to work, most evoke a new media non-hierarchical ethos in which journalists are given the freedom to write about topics that interest them and develop their own ‘projects’. Unlike the ‘missing manuals’ for discussion administration these kinds of accounts are often rendered public, for example in the profiles of journalists on the newspaper’s website.

Hence a second act of (con)textualisation came from extracting all the verbal formulae found in journalists’ public self-presentations. I wanted to see how they described what they did – how their own personalised job descriptions narrate certain actions and/or transformations that they wish or consent to be held professionally accountable for, and how they connect themselves to occupational roles, tasks, tools and dispositifs, in short how they inhabit the world of work. The website presentations are usually brief texts which, on the surface, do not contain much contextual information, typically narrating a personal trajectory without explaining its transitions. Nevertheless they can be read contextually if we view verbs as ‘tests’ – tests of what it is possible to do and the transformations it is possible to accomplish from the field position the employee occupies. Hence they often point to the constraining and enabling conditions in which they work. What emerges is a second list – this time a vocabulary that scripts the ostensive repertoire of a media organisation (what it can or should do) and gives an account of professionals’ relationship to their jobs, the organisation and the public by enumerating the knowledge-processing functions and transformations they perform. It is a sort of map of the organisational field as a space of possibles which could be used to compare how actors occupying different positions (e.g. age, seniority, profession, department) choose to publicly contextualise their place within the field.

I discovered interesting differences in how people narrativise their own agency. Some say they ‘work’ in a particular department, or ‘are responsible for’ certain tasks, whereas others ‘contribute’, ‘comment’, ‘try to explain’, ‘devote myself’ to a particular theme, or simply ‘write’. The former type of formulation distances the individual from the role, gives agency to the organisation and invokes context as a strong constraint (almost disavowing individual agency for knowledge creation); the latter kind asserts a professional or vocational identity, gives agency to the individual and invokes organisational context as an enabler. Perhaps surprisingly, in view of the newspaper’s image as a bottom-up, enabling organisation, the two types of formulation were equally common.

One of the aims of this exercise was also to make visible absences and discrepancies – the activities that get done but not claimed; the tasks that cannot be put to the test and be rendered accountable in this particular public space. Tellingly, eParticipation functions such as discussion administration fall under this cloak of silence, evidently because they are accorded little professional prestige but possibly also because the very anonymity of the discussion administrators is felt to be intrinsic to their public authority. Whatever the reasons, eParticipation is decontextualized on the public-facing website of the media organisation: as if it happens without anyone’s agency; as if it were a continuous flow of knowledge without any active points of intervention or transformation by specific agents, the effect being to depoliticise (to limit the disputability of) the tests that are applied to the knowledge produced in online discussion spaces.

 Consequences for research design

Starting from the proposition that we can define context pragmatically as « l’ensemble des procédés utilisés par les textes ou les discours eux-mêmes pour situer leur propos » (Chateauraynaud 2003: 193), there is a double research task: to recognise and follow the relevant linguistic markers of contextualising speech acts in ‘indigenous’ texts, and to generate additional texts through interventions that invoke a justificatory discursive register. Contextualisation hence becomes an integral research technique, one which I am arguing is essential to this kind of research enterprise, and which deliberately mimics the justificatory contextualisation of participants in conflicts over the ostensive aspects of organisational routines. It provides the capacity to explore what Musil called a sense of the possible (Chateauraynaud 2003: 203) – « la faculté de penser tout ce qui pourrait être ‘aussi bien’ et de ne pas accorder plus d’importance a ce qui est qu’à ce qui n’est pas ». It mediates the staging of controversies intended to increase participants’ own reflexivity with respect to the ostensive aspects of routines, since « If people who enact routines reflect on what they are doing, they may decide to alter what they do in future iterations of the routine. » (Pentland & Feldman 2005: 809) The intention is to avoid the kind of situation in which (re)contextualisation acts as a way of reducing reflexivity (relativising by multiplying the possible measures, finding excuses why proposals are not applicable to this particular situation) and instead enact one in which contextualisation heightens reflexivity by launching a game in which partners consent to explore staged controversies constructively in a space that is attributed the licence to innovate by virtue of the adoption of a justificatory discursive register: a space for the performance and testing of ostensive scripts.

The most significant consequence for my research design is to overturn its overall temporal dynamic. Instead of starting with general landscape mapping interviews with ‘authorities’ in the field (in order to both ‘define the context’ and identify participants for the ethnographic phase) I am starting from particular tasks that need doing (e.g. administering online discussion) and tracing the network of actants enrolled. I still plan to carry out some high-level interviews, but they have to come at the end of the study. They will in fact retain a sort of landscape mapping purpose, but will be about mapping, and testing, the contextualisations enacted by the research intervention. Did they find any takers? Did they help actors make sense of their situation?  Did any of the ostensive collections, categories and collective identities that emerged from staged controversies begin to be performed? Did the core competences enumerated collaboratively become respectable, claimable or evaluatable in the everyday life of the organisation? This type of landscape mapping will have to use ethnographic techniques as well as interviews to assess whether routine indigenous acts of justification begin to mobilise any of the contextualising associations proposed. It is not a passive test of what belongs intrinsically to the field, but an active one of what survived and what resisted an attempted rearrangement; of how, if at all, expertise was reconfigured within the organisational landscape; of whether the centre of calculation changed position.

For Flyvbjerg (2006), the value of a case study depends less on good selection than on good justification and accounting, and the best reasons for the selection of the case will almost inevitably (and not at all cynically) occur to you retrospectively. He advocates justifying and accounting by linking data with heterogeneous knowledge ‘externals’ (e.g. theoretical resources from several different disciplines) in order to aid third-party interpretation and keep the case ‘open’, permitting several answers to the question ‘what is this a case of?’ I extend this idea by arguing for justification and accounting to begin in the research situation itself, taking inspiration from and intervening in the indigenous forms of these knowledge practices that are intrinsic to organisational learning.

References

Abbott, A. (1988) The System of Professions: An Essay in the Division of Expert Labor. Chicago: University of Chicago Press.

Chateauraynaud, F. (2003) Prospéro. Une technologie littéraire pour les sciences humaines. Paris: CNRS ÉDITIONS.

Estienne, Y. (2007) Le journalisme après Internet. Paris: L’Harmattan.

Flyvbjerg, B. (2006) ‘Five misunderstandings about case-study research’ Qualitative Inquiry 12(2): 219-45.

Pentland, B. & Feldman, M. (2005) ‘Organizational routines as a unit of analysis’ Industrial and Corporate Change 14(5): 793-815.

Périlleux T (2005) ‘Le déni de l’évaluation’ Travailler 13(1): 113-34.

Smith, S., Ward, V. & Kabele, J. (2014) ‘Critically evaluating collaborative research: Why is it difficult to extend truth tests to reality tests?’ Social Science Information DOI: 10.1177/0539018414525292


Laisser un commentaire

Votre adresse de messagerie ne sera pas publiée.