Background and Framework

2.1 Agile

2.1.1 What is Agile methodology?

Set off by the failures of traditional approaches to managing software development projects, a group of practitioners motivated to find a better way to project management came up with an Agile philosophy in 2001 (Waters, 2007). Their claim is that they have uncovered an efficient and effective way of software development using Agile principles as clearly stated in their manifesto.

The different approaches that aligned themselves to the Agile manifesto became known as Agile methodologies. The most popular methodologies are Dynamic Systems Development Method (DSDM), Scrum and Extreme Programming (XP). According to Waters, Scrum and XP are easier to implement because they tackle different aspects of software development projects (Waters, 2007).

Scrum[1] became popular because of the way it introduces Agility using a simple and flexible approach that addresses many of management difficulties during project execution. It emphasizes practical feedback, team self-management, and focus on building product increments within short iterations (Agile Methodology, 2008). In 1974, Edmonds published a paper on adaptive software development process phrasing such software development practices as ‘lightweight’. Early implementations of lightweight methods include Scrum (circa 1995) amongst others. These are now typically referred to as Agile methodologies (OMICS, 2014).

A survey completed by Ambler in March 2006 collected responses from a large number of software developers who were using both traditional and Agile methods. The statistical analysis of the data collected reveals that Agile methods improve results of software development projects in terms of quality, satisfaction, and productivity. Parsons, Ryu and Lal[2] also suggest that using a combination of Agile methods can be particularly beneficial. The adoption of at least one Agile method improves the outcomes of quality, satisfaction, and productivity over the use of non-Agile methods, without a significant increase in cost (2007). Another quantitative research of Agile methodologies concluded that the greater the iterative (Agile) approach reported, the higher the reported project success.

2.1.2 Agile and the organization

Schwaber and Beedle[3] present practical evidence of productivity gains, higher adaptability of teams, reduced uncertainty and greater human comfort for projects that used Agile Scrum practices diligently during projects. Agile approaches reduce risk and uncertainty by making everything visible at an early stage and by allowing adjustments to be made as early as possible. The authors have several years of experience managing projects using different approaches, in fact they extend the Agile Scrum approach to managerial and non-technical organizational issues. The new methodology can be used to re-engineer an organization so that it is more productive. They argue about organizations that are never optimized for productivity and present a case which is very common in large organizations such as the public service. In this regard, Agile methodologies such as Scrum helps organizations ‘to optimise, freeing employees to do their best and removing impediments to productivity’ (Schwaber and Beedle, 2002:143).

One of the objectives of the Scrum methodology is obstacles removal. It offers organizations the opportunity to identify obstacles to productivity and which are flagged until they are resolved. Another case revealed by Schwaber and Beedle which relates to obstacles (referred to as impediments) identification, helped management become aware of organizational encroachment on productivity due to excessive bureaucracy.  Scrum fundamental values are applicable to any work environment and any type of organization and in fact, even to one’s personal life.

2.1.3 Agile operations

Sutherland argues that Agile Scrum can actually be used to improve the work output of any team or business and that it works everywhere (2012). However, empirical evidence to prove this assertion is very scarce. Further study of the roots of Agile Scrum shows a close link to lean manufacturing which is an industry that isn’t exactly analogous to software development. As Sutherland explains it, the basic idea behind the methodology is to inspire teams to work better and develop higher quality outputs. The aim of Agile operations is to help an organization respond quickly to the changing demand in order to retain current and increase market share (Greasley, 2008).

At OpenView Labs[4] where Sutherland is a senior advisor, a collective group of industry experts published a case of the Agile Scrum methodology for non-technical environments. Since the predominant objective of this approach is speed, efficiency, and improved productivity, these experts argue that customer service teams using the methodology for instance could translate into more efficient service response. But ultimately success depends on whether you can get people to embrace the new methodology. According to Sutherland, convincing non-technical teams to use Scrum is the real challenge (McDonald, 2016).

There are five key ways that the methodology can support non-software-focused professionals to improve efficiency and effectiveness:

1. Comprehend how tasks fit into the bigger picture and change the focus on the actions the team performs to execute those tasks while keeping in mind the ultimate goal,
2. Although, defined roles within an organization are ultimately held accountable for a finished project or product/service, the methodology encourages autonomy within a team structure,
3. Apart from the ultimate goal, the team is responsible to deliver according to the set sub-goals and specifics. This gives the team’s manager a view of what to expect and when,
4. The methodology necessitates open communication and tracking, it lays out a logical path for getting things done,
5. To leave room for the unexpected and to define an organizational role to look out for obstacles and handle them to help the team reaching its goals.

In addition, these experts argue about the challenges of work iterations, the definition of what exactly the product is, and to develop a functional team for non-technical jobs or tasks (Sutherland, 2012). Cain argues that although the above points look common sense, how many of them do teams routinely follow? Cain re-phrases the above into Plan and Prioritize, Divide Projects into Manageable Tasks, Multi-tasking Isn’t Always a Good Thing, Focus on Getting Things Done, and Hold Yourself Accountable (2012). Denning uses the Agile Scrum terminology to differentiate from the roles and practices of traditional management, and shows how the new underlying managerial principles and practices have roots in many different fields and apply to all sectors of the economy (2010).

In a Harvard Business Review, authors Rigby, Sutherland and Takeuchi argue that Agile methodologies are an alternative to command-and-control-style management offering a number of major benefits, all of which have been studied and documented according to the authors. Team productivity and employee satisfaction are enhanced while, repetitive planning and excessive documentation are eliminated. By engaging team members from multiple disciplines as collaborative peers, the methodology builds mutual trust and respect between and within organizational teams (2016). The authors refer to first-hand experiences that demonstrate a 60% increase of productivity and response to customer input of stable teams when compared to teams that rotate members. Activities that are very well suited include strategy development and resource allocation, stimulating innovative ideas, and improving institutional collaboration (Rigby, Sutherland and Takeuchi, 2016).

2.1.4 Agile – a culture change

Denning argues that Agile is a mindset, not a methodology and to get the benefits of Agile, managers not only have to “do Agile” but they have to “be Agile”. When Agile is viewed and implemented merely as a methodology within the existing management framework, Agile can become counterproductive and what some have labelled it as “fake Agile”. He criticises Rigby et al.’s work as it induces readers to “do Agile”, without making clear that sustainable gains only come from “being Agile,” and embracing Agile as a mindset and a culture (2016). Change must be owned by line departments and take initiatives of their own to make it happen. Central senior-level executives  must avoid assuming sole ownership of reform (Polidano, 2002).

From a practical perspective, this study has an implied objective of using Agile as a framework for change rather than making the public service become Agile (Maza and Benz, 2016), more as a means to an end, than an end in itself. A large part of being Agile is about delivering what government needs in more innovative ways (Mitchell and Moore, 2014).

2.2 Agile in the Public Service

Reflecting on the Australian State Services Authority and Demos[5] paper (Parker, Bartlett 2007), Professor Gallop links agility to three characteristics: i) A public service working effectively in a world of constant and sometimes rapid change; ii) facing complex problems in an uncertain environment; and iii) a public service that must handle crises, be they natural or human induced (2007). When delivering services Agile governments need to introduce suitable methods of consultation and engagement into their practices, however, Professor Gallop argues about the importance to distinguish between the different roles of government (for example: rule-making, enforcement and service delivery). Moreover, the need to include the political levels more directly into the discussion. Politicians are not only involved in one of the activities (rule-making) but they are crucial when it comes to the implementation of rules and associated activities (Parker and Bartlett, 2007).

2.2.1 Agile implementations at technical levels

Two Agile coaches describe how they transformed an undisclosed federal government entity into using Agile Scrum principles, practices and techniques. A process that helped this government entity redefines how it operates, while moving from a matrix to a team centric structure (Raines and Neher, 2015). Another instance of Agile software deployment for US National Institutes of Health yielded very positive results. The unique challenges of the regulated and highly political environment required the adaptation of some of the Agile practices.  Agile practices have helped all stakeholders achieve better communications and become more committed to work. Daily visibility into progress gave the team more freedom to make decisions (Upender, 2005).

The Danish governmental Business Authority adopted Agile methods, processes and approaches into its traditional ways of working. This authority achieved an increase of about 60% in productivity and customers’ response has dropped from 16 minutes to 5 minutes (Lundqvist and Olesen, 2016). The United Kingdom government has taken Agile methodologies for software projects a step further and set a government-wide policy – ‘Digital by Default Service Standard[6]’. Apart from the well-defined Agile practices and methods, the notable criteria relevant to operations include the requirement to have multidisciplinary teams, the capability to iterate a service quickly based on feedback, and the requirement to test the final product with the Minister as the ultimate owner (Gov.UK, 2016).

2.2.2 Agile implementations at operational levels

AT Kearney[7] conducts regular research known as the Agile Government Index. The Index is calculated by combining survey data of selected senior officials from a range of core agencies within governments and quantitative data from reliable sources such as, the World Bank and World Health organizations. Earlier research indicates that Agile public sector organisations benefit from higher levels of efficiency, higher employee satisfaction and positive feedback from citizens. AT Kearney first study of 2003 found that Agile public sector agencies experienced a 53 per cent productivity increase, a 38 per cent increase in employee satisfaction and a 31 per cent rise in customer satisfaction (Zibret et al., 2014).

There needs to be a shift from the command and control way of operations to innovation and collaboration. Governments are gradually identifying new ways to drive improvements in public services which contradict traditional methods such as, specifying targets from the top, putting more money and pushing for ever higher standards. Practical improvements will only come by permitting public sector agencies to innovate from within, allowing them to develop their own new methods that meet the needs of citizens and end users. There needs to be a change in service design linking providers and users in co-designing services. Governments operate in unpredictable environments and therefore cannot rely on static modes of operations. The public service needs to be able to adapt its structures and processes to care for the short and medium term trends and challenges. Strategic adaptation means that public sector agencies are able to adjust their operations as the environment changes (Parker and Bartlett, 2007).

A new methodology is emerging which underlines the need for collaboration, participation, innovation and a focus on outcomes. The concept of Agile methodology provides a useful framework for drawing together these qualities, and enabling the public service to meet the challenges of today and the future (Mannix and Peterson, 2003).

Over the course of 2014, the Canadian Public Policy Forum and PwC[8] worked with a panel of senior administrators across the Canadian government to explore the concept of Agility. Following round table discussions and over 45 one-on-one interviews, it was found out that Canadian leaders pushed for Agility in their everyday operations and that 92% of participants believe that Agility is achievable. The barriers that Canadian leaders agreed upon are structural, and are unlikely to change in the short-to medium-term, simply because the accountability measures are necessary for the expenditure of public funds. Barriers that are based on tradition or past practices can be overcome and the first pre-condition for Agility is that change has to begin at the top. The case studies profiled offered evidence that innovative change did happen within the Canadian public sector and one major key reason was due to providing staff with more capacity to take risks and redesigning business rules. Finally, this study concluded that there are ways to ensure that checks and balances are in place, while at the same time a public sector entity is responsive, creative and adaptable (Mitchell and Moore, 2014).

2.3 Public service operational context

Professor Gallop, who administered a range of political and social reforms as the Premier of Western Australia from 2001 to 2006, in 2007 argued about the incomprehensible range of objectives which many public servants are expected to follow, both individually and collectively. Public servants are expected to be fully accountable and yet are asked to be creative and innovative, and to perform to particular targets and at the same time to be agile and flexible in the way they operate (2007).

2.3.1 Environment, goals and objectives

Room argues that the assumptions of stable environments and well-defined policy problems are not appropriate for today’s public service reality. Room’s proposed toolkit for the practice of Agile-policy making tries to deal with the continuous change of objectives for institutions. Policy makers need to engage using practical methods of operations and communications and follow an iterative process that decomposes the complex process of policy making into manageable sub-processes. A process made up of continuous modification and evolvement of elements, adapting them to specific situations as he refers to as ‘iterative assessment and prioritization of objectives’ (2011c:240).

Morgan and Murgatroyd highlight the problems associated with both time and timing of work in the public service. They argue that until the normal practices of annual cycles for resourcing, objectives’ setting and performance management are changed, then time will be a barrier to the effective implementation of Total Quality Management[9] (1994a). Adaptive capacity needs experimentation and innovation by permitting flexibility and risk taking, encouraging approaches to emerging challenges (Dyer and Shafer, 2003).

Problems resulting from poor clarification of roles and objectives are a major cause of employee frustration and confusion (Buckingham and Coffman, 1999). Increasing levels of authority in terms of decision making or independence is a good way to reward and develop subordinates. Delegation of responsibilities must be preceded with clear specifications of the new responsibilities, scope of authority, and rewards and punishments for not meeting standards or using authority well. Clear reporting responsibilities and structures must be in place, and how to relay information and inform people about accomplishments (Van Wart, 2012c). Inspiring involves providing encouragement to work for group and organizational goals regardless of personal benefits. Frequent and timely recognition is more motivating than recognition long after the fact (Van Wart, 2012b).

2.3.2 Structures and management

The right to manage is normally understood as the right to tell people what to do and expect from them. If it is not about the right to develop staff, encourage commitment, and form teams to instil loyalty, then, it is a management style that implies hierarchy, authority and fear (Flynn, 2007). Accountability, the notion of multiple stakeholders and competing domains is a ‘complex phenomenon’ in the public sector (Isaac-Henry et al., 1997:293). A preference for more specialized, lean, flat and autonomous organizational structures is a key element of the ‘New Public Management’ (Bekkers et al., 2011). Additionally, an important shift is towards the focus on the process of governing instead of the structures of government. These processes concern negotiation, concentration and cooperation rather than the traditional process of command and control (Kersbergen and Waarden, 2004).

Many public institutional settings involve simple rules of ‘myopic administration’ which characterise systems of hierarchical coordination. Institutional minimalism and hierarchical direction presuppose myopia (Room, 2011b:70). Policy interventions that interfere with self-organization are misguided and are likely to reduce the outcome (Room, 2011a:311). Human Resources Management (HRM) practices should aim to enable the employee’s identification with organizational objectives. The application of this approach is supported by empirical studies suggesting that the implementation of such HRM systems leads to better production (Delaney and Huselid, 1996). From an HR policy perspective, all employees must at all times be clear about what outcomes they owe, to whom and by when (Dyer and Shafer, 2003).

Advocates of managerialism argue that proper management should derive its existence from the practices of the private sector. Practices that highlight output measures, shift to desegregated units and decentralize management, and emphasize the role of key stakeholders. Bureaucracy can be lessened by setting in objectives, responsibilities and expectations, and the separation of operational matters from strategic ones. Therefore, leaving units to concentrate on operational matters that achieve technical efficiency (Isaac-Henry et al., 1997). This can be defined as the efficiency that indicates the rates of input to outputs, rather than the social relevance of a government activity (Kang, 2008).

2.3.3 Employees motivation

An empirical study[10] among 9852 civil servants at the Swiss municipal level attempted to investigate public sector motivation in relationship with intrinsic and extrinsic variables (Appendix B). The empirical results stemming from this study remain questionable as how generalizable are they to environments outside the Swiss municipals context. Nonetheless, this study confirms that material incentives are poor predictors of work motivation in the public sector while, socio-relational factors such as recognition from colleagues and superiors, reliance and interactions with colleagues are good predictors of work motivation (Anderfuhren-Biget, 2010). When workers are able to grow through their work, work becomes part of one’s identity (Berman, 2008) while, empowering employees to take self-initiative is boosted when leaders push decision-making activities down the corporate ladder, and delayering management (Haines, 2009).

There are some organizational cultures which incapacitate front-line staff rather than enabling them. A decentralized and participative culture is most likely to be compatible with a successful implementation (Isaac-Henry, Painter and Barnes, 1997). Although, culture is not easily changed, it can be modified to enhance socialization and team development (Lundberg, 2008). The balance between central control and local responsibility needs to be reassessed in order to achieve a successful public service provision but in a way that maintains the controls necessary to uphold standards and enforce accountability (Minoque et al., 1998).

According to Flynn, an important aspect of the service relationship is the way the organization treats its own workforce (1990). Effective service delivery depends crucially upon successful methods of managing, rewarding and motivating human resources (Duncan, 1992). A survey[11] conducted over a representative sample of leaders from 1502 British public institutions shows that employees are the most important source of innovation with a strong correlation between engagement/involvement of employees and higher levels of innovation.

2.3.4 Individuals’ vs teams performance

In view of leadership skills or the lack thereof in the public service, self-managed teams provide a means of team leadership. On one hand, a healthy self-managed team binds itself with the work, distribute tasks based on talent and interest, and apply flexibility and innovation while, on the other hand a poorly functioning self-managed team induce frustration among members (Van Wart, 2012a). The ideal conditions that must exist for self-managed teams to perform well include a shared goal, shared accountability and benefits, complementary skills and an appropriate number of team members. Relatively small number of team members allows for better interaction and trust (Katzenbach and Smith, 1993). Self-managing work teams are a core component of TQM (Morgan and Murgatroyd, 1994a).

Team-based rewards may also be preferred to individual compensation schemes, especially in contexts where co-operation is important for the outcome of the organisation or where only aggregate measures of performance are available (Burgess and Ratto, 2003). Also, when tasks are reduced to those set targets given on individual basis (‘management by numbers’), other services not included in the targets might be neglected. Public service bodies may be diverted from their strategic purpose by performance targets (Isaac-Henry et al., 1997:14). The link between individual goals and agendas, and objectives of the organization/department are a main concern for TQM in the public sector (Morgan and Murgatroyd, 1994b).

Common types of problems that are commonly encountered by teams, range from overbearing and dominating members to reluctant and passive members. Furthermore, a common problem is that teams lose track of their purpose or goals and eventually risk of accomplishing the wrong thing (Scholtes, 1993). TQM research within the public sector shows that when teams really become involved in the management of their own work and eventually take complete control including the definition of tasks and work processes, they are in the right path to the development of empowered intact work teams through involvement & ownership (Morgan and Murgatroyd, 1994b). Any organization requires the synchronised efforts of more than one person and the contribution of any single person is strongly dependent upon the performance of the team (Goldratt, 2014a).

2.3.5 Operational performance

Sometimes operations planning may be more important than strategies, a term often used for ongoing prioritization and scheduling activities (Van Wart, 2012c). Performance management is used to describe the range of processes, techniques and methods to achieve an improvement in the productivity and quality of the relationship between inputs, activities, outputs and outcomes in public service organizations. It is complex since goals are rarely clear or consensual given the multiple constituencies of stakeholders involved and the complex environments of public service delivery. Therefore, it requires clear objectives with an emphasis upon outputs and outcomes, the need of having both appropriate technical and procedural practices, and supportive cultural and attitudinal characteristics within the organization (Isaac-Henry et al., 1997). Osborne and Brown argue that while formal and systematic procedures create consistency of action and predictable responses, they also routinize and stagnate activities. The authors suggest that change must be continuously moving through a cycle of initiation, uncertainty, transformation and routinization (2005).

Research within the UK government suggests that the ideal performance reward scheme for public service agencies depends ultimately on the type and objectives of the organization. However, there is a strong correlation between ‘task assignments and work organization’ and better performance. This research also remarks that performance related pay is more problematic in the public sector than in the private sector due to aspects like multi-tasking, multiple principals, the difficulty of defining and measuring output, and the issue of motivation of workers (Burgess and Ratto, 2003). A short study about the performance management program conducted across the Maltese public service reveals that the current system lacks objectivity, consistency and credibility (Bonnici, 2010). While, the program provides a formal structure for communicating work expectations, organizational goals and other job-related feedback, such activity only happens once or twice a year (Spiteri Gingell, 1995). Linking performance management reviews to rewards and sanctions may contribute to improved productivity (Bonnici, 2010).

The Maltese public service has recently embarked on a new performance programme which apart from individual performance and development, caters for benchmarks for quality customer care and departmental performance. A performance rating plan for departments providing external customer services is aimed at achieving efficiency, and uniformity of performance through standards and well defined goals. The introduction of new dimensions of quality service should create a positive sense of teamwork when all elements work together towards a common goal (Cassar, 2016).

2.3.6 Maltese public service context

Directive 6 issued by the Principle Permanent Secretary dictates a framework for ensuring stakeholder involvement in decision making[12]. The framework comprises a procedure for government entities to follow when conducting consultation exercises with stakeholders. An important objective of Directive 6 is to ensure an adequate and proper consultation process in order to gather the views expressed by relevant stakeholders (Directive 6, 2011). Similarly, Agile philosophy values customer collaboration and communications (interactions) more than other attributes. In an effort to achieve improved efficiency and effectiveness of the HRM function within the public service, Directive 10 mandates the preparation of three year business and HR plans for all public service entities (Directive 10, 2015). Although, the Directive suggests that plans are to be revised annually to allow for the necessary adjustments of procedures and processes, such long-term planning may be beneficial to financial predictability but may hinder entities adaptability (and Agility) to today’s fast changing environment.

During the 2016 Public Service Week, the Maltese Prime Minister claimed that despite the Maltese government ranks as a top leader of online services implementations, the use of such services is very low. Various initiatives to improve the delivery of quality services and back-end office operations were launched. The Principle Permanent Secretary claimed that regardless of these initiatives, Malta still ranks very low in the World Bank group’s ‘Doing business 2016: measuring regulatory quality and efficiency’ (Editorial, 2016). In spite of the 250 ‘simplification measures’ to improve public service procedures, the outcome of certain initiatives is still unsuccessful and shortcomings were still being repeated. During the 2016 annual Public Service Conference, the term simplification was at the top of the agenda and further simplifications are needed especially for online services, in order to continue the path towards a better and smoother service offered to citizens (Martin, 2016).

2.4 Conclusion

The literature review shows that Agile can be applied to operations in non-technical areas, however, relevant empirical evidence is very limited. The methodology can be modified and has various derivatives, for instance the United Kingdom government adopted Agile methodology for all software projects developed within and by the government. The policy remained faithful to the standard practices of Agile, and added extra attributes that extended the scope of software development within the UK government. Other governments such as the Australian and Canadian debated and introduced the concept of Agile government at strategic and policy levels, defining high level goals without defining details relevant to the day-to-day operations to achieve those goals. To date, no references of Agile deployments have been identified within the Maltese government.

Findings from the literature will be used to develop a conceptual or theoretical framework for this study. The following chapter will explain the method used to conduct the investigation in order to gain an understanding of how Agile concepts could impact the Maltese public service.

[1] The name Scrum is taken from the Rugby game. It is the event where players from both teams form a pack against each other in an attempt to gain possession of the ball.

[2] In their research the authors executed a quantitative analyses of the data set made available by Ambler.

[3] Schwaber is cofounder of the Agile Scrum methodology and Beedle is one of the first Agile Scrum adopters.

[4] OpenView Labs is an editorial site publishing web content in areas of product development, customer success, marketing, finance and operations, amongst other topics.

[5] Demos encourage political contribution by concentrating on the way that personal experiences interact with impersonal, national and global forces.

[6] The U.K. Digital Service Standard is a set of criteria to help government create and run good digital services. All public facing transactional services must use and meet the standard.

[7] A.T. Kearney is a leading global management consulting firm that combines commercial and public-sector expertise to address governments’ most challenging issues.

[8] PricewaterhouseCoopers (PwC) is a multinational professional services network and one of the largest professional services firms in the world, known as the Big Four audit firms.

[9] Total Quality Management is a system of management based on the principle that every member of staff must be committed to maintaining high standards of work in every aspect of a company’s operations.

[10] Response rate of 38.1%

[11] Based on stratified cluster sampling with a response rate of 51%

[12] The Public Administration Act empowers the Principle Permanent Secretary to issue orders known as Directives (article 38 (3)).