In this section I present an explanation of the methods adopted during the investigation. The purpose of this section is to shed light on the inquiry and the process used (Kaplan, 1998). This qualitative exploratory research aims to provide insights into potentially new public administration operational methodologies by using a deductive approach. That is, theoretical propositions are derived from existing theory to devise an analytical framework which will inform my analysis and recommendations (Saunders, Lewis and Thornhill, 2009:489).
The deductive approach applied in my qualitative analysis devised theoretical propositions prior to data collection (Saunders, p.578). In other words, I utilized Agile reviewed concepts to develop a conceptual framework and then tested the adequacy of this framework as a means to explain my findings. Saunders refers to the prediction of a pattern of outcomes as Pattern Matching (2012:579) as depicted in figure 3.1 below:
Based on theoretical propositions I proposed a number of related outcomes that I expect to find as a result of an Agile implementation in the Public Service (Saunders, 2012:579). However, to avoid validity issues, I had to assess whether public servants still face the operational issues or context explored in the literature review (Section 2.3). That is, the researched environment needs to be assessed before and after the simulated Agile implementation. Therefore, the deductive process as depicted in figure 3.1 above, was modified to include this additional step as shown in figure 3.2 below. The development of the conceptual framework was a lengthy process and involved a number of iterations before it was completed (Appendix G ).
The initial categories and elements that formed the structure of the qualitative interview were derived from the conceptual framework. Saunders et al. refer to this method of data categorization as concept-driven category (2012:557). The advanced definition of categories help show the strength of participants’ opinion that may in some instances lead to further subdivision or re-arrangement of categories. Additionally, the concept-driven categories were designed in a way to form one coherent set made up of three parts – Part A, B and C (Table 3.1 and Appendix F). The set and the subsequent parts provided the analytical framework used for my analysis (Saunders et al, 2012:558).
In qualitative research, purposive sampling is frequently used because it allows for cases to be chosen on the basis of the researcher judgement (Miles and Huberman, 1994) to help him/her illustrate a typical case. In contrast to quantitative methods, a relatively small number of informative cases is used and still considered adequate to meet the research objectives (Neuman, 2005). It provides an illustrative profile that is considered representative, albeit not statistically (Symon and Cassell, 2012). I am aware that a small sample will prevent me from making statistical inferences about the public service, but it will make it possible to draw generalisations provided that the cases chosen were appropriate (Saunders et al., 2009).
In terms of the operational characteristics being studied in this research one Ministry is a truthful illustration of other ministries within the public service. Although, the sample taken from the Ministry cannot be used to make statistical inferences about the public service as a whole, the sample outcome remains illustrative for other similar contexts. Excluding public sector agencies which may fall under the remit of the Ministry and political level sections such as, the Minister’s secretariat we find that similar characteristics and methods of operations prevail resulting in a fairly homogenous environment. The research questions that are being addressed are specific to the characteristics of the particular group of interest, mainly service operations and administration within the Ministry and such homogeneity is fairly consistent across all Ministries within the public service.
Nonetheless, heterogeneous features of purposive sampling have enabled this study gain greater insights into the phenomenon by looking at it from all angles (Symon and Cassell, 2012). In fact, this aided my judgement not to ignore potential sections with slightly diverse characteristics that may exist within service operations and administration of the Ministry. To ensure that cases were included from all key operational activities and across all levels of the hierarchy, I identified five key groups or sections within the Ministry (table 3.2).
Approval to conduct the interviews with civil servants from the designated sections was requested and obtained from the People and Standards Division (PSD) of the government. An initial email request including an introduction letter from my supervisor and an information sheet for participants was sent to the ministry’s permanent secretary (Appendix C). Prior to final approval the People and Standards Division requested clearance of the University’s Ethics Committee (Appendix D). The face-to-face interviews were conducted in secluded areas within each section so that participants could feel at ease and can focus on their replies. Interviews were not recorded and all participants accepted to participate and signed a formal letter of consent (Appendix E).
Participants’ replies were recorded on sheets using just a sequence number to indicate the number of interviews performed. There was no mapping between consent forms signed and data sheets to ensure participants anonymity. Saunders recommends the use of identifiers to maintain anonymity and facilitate retrieval of data generated during the interviews. (Saunders et al, 2009:488). Data was transcribed at the end of each day so that I could assess it for saturation.
The semi-structure interview format was done in a way that resembles a three phase approach (Appendix F). The first part of the interview asked participants to confirm or deny a number of operational problems derived from the literature review and which Agile methods aim to solve. In the second part, participants were involved in a brief explanation of the concepts to envisage an Agile working environment. This part of the interview made participants answer questions related to operations from an Agile perspective. Finally, the last part focused on Agile practicality and when time permitted, a short exchange of further observations was conducted. In fact, this last part gave rise to a number of very stimulating arguments.
A carefully selected purposive sample of 12 participants is likely to suffice when the research aim is to understand commonalities within a fairly homogenous population (Guest, Bunce and Johnson, 2006). Cresswell (2007) suggests between 25 and 30 participants if the target population is more heterogeneous. Additionally, Guest et al. (2006) advice that two samples of twelve participants for a comparison of two homogenous groups, are likely to be sufficient. Bertaux (1981) suggests up to 15 participants, whilst Becker (2012) notes that one participant may be sufficient for some purposes. Kuzel (1992) recommends that where the population of interest is homogeneous, 6 to 8 participants are likely to be sufficient, whereas for heterogeneous populations 12 to 20 participants are likely to be needed. Much of this is summarized by Saunders (Symon and Cassell, 2012), who notes a range of 4 to 12 participants as likely to be sufficient when chosen from populations considered homogeneous, and 12 to 30 participants when chosen from populations considered heterogeneous.
In view of the possibility that marginally different modes of operations may exist within the Ministry, I considered literature referring to both heterogeneous and homogenous contexts, and decided to target a sample size of 30 participants. On the other hand, considering the novelty of the research idea within the public service I anticipated that the semi-structured interviews may take longer than planned and 30 cases may be too time-consuming. In addition the literature suggests that in the open-ended nature of qualitative research, ideally data collection should continue until saturation (Morse, 1994) or informal redundancy (Lincoln and Guba, 1985) is reached. While, the sample size was set to 30 cases, I observed the data being collected and stopped at a point where no new information or themes were observed.
Morse (1994: 147) states that saturation is the key to excellent qualitative work. This technique is dependent upon data being collected until no new themes or information are observed, that is until saturation is reached. Francis et al. (2010) propose ten interviews as a guide of when to start looking for saturation followed by three more to substantiate it before stopping. However, this technique introduces various challenges such as, it is often necessary to have an idea of the number of participants on the outset and that the saturation threshold is hard to detect while data collection is under way (Safman and Sobal, 2004). Where the population is homogenous data saturation is likely to be reached having collected data from between 4 and 12 participants. In contrast, for heterogeneous populations, data are likely to need to be collected from between 12 and 30 participants before saturation is reached.
The split of the Ministry into four areas forming the sample frame was to make sure that key areas of operations within the Ministry were not excluded. Irrespective of public servant’s grades within the hierarchy, employees work together individually and as teams. Nonetheless, I asked Head of Sections to select a mix of different grades for the interviews, on the condition that each case has more than 5 years of public service work experience. Five years of work experience are considered as adequate given that many private organizations deploy 3 to 5 years job rotation schemes.
The selection of the number of cases from each section was taken proportionally to the number of employees in that respective section. From a total sample frame of 90, 20 cases were selected from operations, 3 from administration, 3 from internal services, 2 from external services, and 2 from a specific function.
This distribution satisfies the size of the sample (30 cases) derived from the literature review about purposive non-probability sampling. The literature recommends estimates of around 30 participants when chosen from a single organization, and around 50 participants when chosen from multiple organizations (Symon and Cassell, 2012).
All cases interviewed had 5 or more years of public service work experience, and the selection of different sections and employee grades within key service operations within the Ministry had in fact made the research more interesting. The overall list of different grades that participated in the research is depicted below:
Irrespective of participants’ grades I observed early saturation throughout the different sections and this confirmed researchers’ studies of a homogenous population. With regards to the support services section of the Ministry which had the largest number of participants, I exceeded the saturation point in order to make sure that I have an adequate number of different grades in the sample. Out of a sample of 30 cases, no significant information was observed after circa 20 participants and therefore, to confirm saturation I performed 3 additional interviews and concluded my data collection process after the 23rd interview as shown in table 3.2.
As already noted, the sample size used prevents me from making statistical inferences about the public service, but given the homogeneity of Ministries it will make it possible to generalize theoretically . Another important constraint was that participants had to adopt a reflective and imaginary approach to answer questions about Agile concepts. Although, most of them seemed very engaged with the concept and related their replies to personal experiences (Wolgemuth, 2014), still the scenario presented was not experienced for real.
Face-to-face interviews were particular useful in gaining a substantial depth and detail of information about the researched topic. These presented a high degree of flexibility, certainty that participants understood the concept and created the necessary trust between both parties during the interview. However, both interviewer and interviewee biases play an important role as described by Crowther and Lancaster (2012). There was only one instance where I noticed that the participant seemed to be knowledgeable about Agile methodologies and felt that the information collected was somewhat prejudiced. Also, the range of different scales among those interviewed did not inflict problems on the information collected, however, I noted that some grades (scales 5, 7, and 11) were more representative.
In this chapter I presented the method and process of enquiry used in my data collection approach. I also provided an in depth explanation of sample sizes in qualitative research as described by various researchers. Different techniques were discussed and a close examination of the chosen method was undertaken. The justification of the chosen method entailed researchers’ recommendations and constraints of the researched topic and target environment. Additionally, I gave a brief overview of the design and authorization process done to gain approval to conduct the interviews. Cases targeted and eventually done on the basis of data saturation were portrayed and the limitations encountered specified. The findings from data collected during the interviews will be presented and discussed in the following chapter.