Project Management Journal December 1995 Page 3

The Role of Project Risk
in Determining
Project Management Approach

By Jean Couillard
University of Ottawa, Ontario, Canada

About the Author

Jean Couillard, Ph.D., is professor of management science at the University of Ottawa, Ottawa, Ontario, Canada, and has taught at the Master in Project Manage- ment Program at Universite du Quebec a Hull, Quebec, Canada. He received his Ph.D. in opera- tions and decision science in 1987, and a M.B.A. in 1979 from the Universite Laval, Quebec, Canada. He has been a project management and transportation consultant for private and public organizations. His research interests include project risk management, project planning and control, and how to a adapt project management approach to project and organization characteris- tics. Dr. Couillard is a member of PMI.

Much of the literature about project management proposes a uniform set of general-purpose tools and methods to manage all kinds of projects. McFarlan [7] suggests that the best management tools and methods vary widely according to project characteristics such as risk. Consequently, there is no universally accepted way to run all projects.

From McFarlan's proposition [7], it could be argued that large, high-risk projects require specific tools, techniques, and resources that differ from those required by small, low-risk projects. For example, it seems reasonable to assign the most experienced project managers to head large, high-risk projects. As well, large, high-risk projects should be more carefully planned, closely monitored, and strictly controlled.

This article investigates the appropriateness of different project management approaches devised to reduce the influence of risk to increase the likelihood of project success. The findings from a field study show that project goal understanding, the level of authority given to the project manager, problem handling by the project team, communication, and team support have a significant influence on project success. These results further confirm literature findings that many success factors center around human relationships.

The findings from this research show that when project risk is not considered the well-known PERT/CPM techniques, project monitoring and control do not have a significant influence on success. However, these results further show that in high-risk projects, these techniques do have a significant influence on success. In high-risk projects, PERT/CPM techniques, which increase the frequency of project monitoring and control, improve the likelihood of project success. Therefore, high-risk projects should be more closely planned, monitored, and controlled.

The study also shows that when technical risk is high, stand-alone project structure has a significant negative influence on technical and cost risk. A possible explanation for this could be found in the fact that technical expertise can be more easily drawn from a matrix structure than is the case with a stand-alone project. Finally, the study shows that when cost and schedule risks are high, an experienced project manager should be selected to head the project.

The research further suggested that project risk should be considered from a strategic and tactical point of view when implementing a project. The "best" management approach to successfully implement a project varies according to the project risk profile.


Research Objectives

Anderson and Narasimhan [2] proposed the concept of project risk assessment as a way of identifying the chance of project success and pointed out the need for the development of appropriate implementation strategies to increase the chance of project success. This article investigates the tools and methods used by project managers to overcome the influence of risk on project success.

The postulate retained in this research is that for a given risk profile there is an appropriate management approach relating to the selection of the project manager as well as the project management tools and methods that will contribute to the likelihood of project success. Therefore, in order to examine empirically the relationship between project risk and the appropriateness of different project management approaches, we propose, based on the work of McFarlan [7], the following hypothesis: the success of a project depends, in a complex fashion, on the project management approach and the project risk.

The data for this investigation were collected as part of a larger survey conducted on success factors in military acquisition projects at the Department of the National Defence (DND) in Canada. The study dealt with military acquisition projects.


Literature Review

Many research studies were undertaken to determine factors influencing project success [1] [3] [4] [5] [8] [9] [10] [12] [13] [14] [16]. A few research studies have empirically addressed the problem of determining the relationship between project characteristics and the appropriatness of different management tools and methods in order to increase the likelihood of project success.

Might and Fisher [9] suggested that formal project control systems have considerably more influence (both positive and negative) on project success in large projects than is the case with smaller projects. The study also showed that different project management control techniques and organizational structures are more or less appropriate depending on the project success measures used. As a consequence, they recommended that structural factors be seriously considered in a strategic sense before determining the appropriate tactical approach to managing a project.

Rubin and Seelig [16] studied the relationship between project characteristics, project managers' characteristics, and project success. They found that organizations tend to select their oldest and most experienced project managers to head large, high-priority projects. In their study, project manager experience had no direct rele- vance or influence on project success but the high priority given to larger projects did have an influence on project success. According to Rubin and Seelig, although organizations tend to select their oldest and most experienced project managers to direct large and high-priority projects, the success of these projects was influenced more by the high priority given to them than by the experience of the project managers.

These studies provide a better understanding of the relationship between project characteristics and the project management approach that gives the greatest probability of success. This research further investigates the relationship between project risk and the appropriateness of different project management tools and methods in order to increase the likelihood of project success. The results of the research are given in the next two sections.


Table 1. Project Success Measures

Tech1 The subjective measure of the technical success relative to the initial requirement (X41) *
Tech2 The subjective measure of the technical success compared to other projects in DND (X42)
Cost The subjective measure of budget over/underrun (X43)
Time The subjective measure of the schedule over/underrun (X44)
Process The level of satisfaction with the process by which the project was managed; a successful project is one that requires minimal conflict and crisis management, Might and Fisher l9l (X45)
Overall The subjective measure of the overall project success (X47)

* X41 is a label assigned to the variable



Table 2. Management Factors

Project Manager Experience

  1. Number of project(s) managed as PM or PD (X) [0,1,...]

  2. Responsibility index (X2) [-1, 0, 1]


Project Management Method

3. Project goals understanding (X6)

4. Level of PM authority and responsibility (X7) [-3, 3]

5. Level of PD authority and responsibility (X8) [-3, 3]

6. Organizational structure (X9) [-3, 3]

7. Senior management involvement X10) [-3, 3]

8. Communication patterns (X11) -3, 3]

9. Problem handling (X14) [-3, 3]

10. Project team support (X15) [-3, 3]

Project Management Tools and Techniques

11. WBS utilization (X23) [0,1]

12. PERT/CPM or CPM utilization (X25) [0,1]

13. C/SCSC utilization (X27) [0,1]

14. Periodic technical reports (X28)

15. Periodic cost reports (X29) [0,1]

16. Periodic schedule reports (X30) [0,1]

17. Frequency of project monitoring (X33) [0, 5]

Findings

In this section the relationship between six project success measures, and 17 project management factors, all described below, is investigated.

Project Success Measures.

Six project success measures proposed in the literature [9] [10] [15] were used in this research. The success measures, given in Table 1, were assessed by project managers (responsible for the conduct of the project) as well as project directors (responsible for the operational requirements of the sponsoring group and acting as clients' representative). According to Pinto and Slevin [15], a large part of the assessment of success relates to the impact of the project upon its intended users, the clients.

A questionnaire was used to obtain, on a seven-point Likert scale, the level of agreement of the respondents with the proposition that their project was a success (see Appendix for a discussion of the research methodology used). A total of 121 valid questionnaires were analyzed in this research.

The management factors that might influence project success were then identified from a literature review and interviews with project managers from DND.

Project Management Approach.

Three groups of management factors were selected to assess the project management approach used. These factors were:

Again, these factors were measured on a seven-point Likert scale in the questionnaire.

Two variables, measuring the project manager's experience, were selected from the study undertaken by Rubin and Seelig [16]: the number of projects previously managed, and a responsibility index. The responsibility index indicates whether the dollar value of the actual project is lower (-1), equal (O), or greater (1) than the dollar value of any previously managed project. In the Rubin and Seelig study, these variables were found to be significantly related to the technical performance of a project.

Eight variables were selected for the second group of factors, the project management method. These variables from the Project Implementation Profile (PIP) model, developed by Pinto and Slevin [14], were found to influence project success.

A first variable assesses the understanding of the project goals by the project team. The second and third variables measure the level of project manager and project director authority, respectively. A fourth variable assesses the level of support from the senior management. A fifth variable relates to the directness of communication in the project. A sixth variable measures the responsiveness of the project team when problems arise. Project team support, a seventh variable, measures the level of cooperation received from the project team. The last variable of the group indicates the organizational structure: matrix (1), semi-matrix (2), or stand-alone (3).

The third group of factors, tools and techniques, includes dichotomic (0,1) variables that indicate if a given project management tool or technique was formally used. The tools and techniques considered are the Work Breakdown Structure, the PERT/CPM, the C/SCSC, periodic technical reports, periodic cost reports, periodic schedule reports, and project monitoring. In the study undertaken by Might [10], these techniques were not directly related to success. However, in his study these techniques in concert with some situational conditions did influence (both positively and negatively) project success.

The management factors selected for this study are given in Table 2. The numbers between brackets indicate the values that can be taken by the corresponding variable.

The Influence of the Project Management Approach on Success. The relationship between project management approach and project success was analyzed using a simple regression model (see Appendix for a description of the statistical method used. The results of the regression analysis are given in Table A3.)

From these data, it was concluded that communication patterns and project goal understanding significantly influence all six measures of success. This result shows the importance of having the project team knowing exactly what is expected from them as well as establishing clear and effective communication within the project team.

The results also indicated that the authority given to the project manager, the support received from the project team, and problems handling by the project team have a significant influence on project success. The authority assigned to the project manager should be clear and sufficient to allow for important project decisions to be made at the project level. Furthermore, the project manager should ensure that the project team is responsive when problems arise. Very few projects are ever completed without encountering any problem. As a consequence, the project team should be informed of the problems that might arise and be prepared to handle them.

From the data in Appendix Table A3, it is interesting to note that, in general, project management tools and techniques do not have a significant contribution to project success. It was also observed that technical and schedule progress reports have a negative effect on time success. Spending time gathering information on the project and writing technical or schedule reports means less time to be spent on the project, thus causing more delays.

These results show that when project risk is not considered many success factors center around human relationships. Effective and simple communication patterns should be established. The basic goals of the project should be established and made clear to the project team. The level of authority assigned to the project manager should be effective and clearly defined. Finally, the project manager should ensure full support from the project team.

In the next section, it will be shown that the project management approach varies according to the risk level involved. In other words, project managers tend to adapt their project management approach to the project's level of risk.

The Influence of Risk on the Project Management Approach. The respondents were asked to assess risk with regard to three project objectives [6]: technical performance (the risk of not meeting the operational requirements); cost (the risk of exceeding the budget); and time (the risk of not meeting the set deadline).

The risk involved in a given project was assessed with regard to these three objectives on a three-point Likert scale (low, medium, and high risk). For the purpose of this research, these aggregate measures were deemed adequate to provide insights into the approach used by project managers to overcome the influence of risk in order to increase the likelihood of success.

Again, regression analysis was used to determine the relationship between the project management approach used and the level of risk. The results of the regression analysis are given in Table A4 of the Appendix. From the data in Table A4, it was observed that a significant relationship exists between risk and the project manager's experi- ence. It seems that the most experienced project managers are generally assigned to the riskier projects. This result is in line with the findings of Rubin and Seelig [16]: organizations tend to select their oldest and most experienced project managers to head large, high-priority projects.

It was also found that PERT/CPM, C/SCSC, and technical reports are more frequently used in high-risk projects. The results indicate that high-risk projects are generally more formally planned, more closely monitored and controlled. The riskier the project, the more formal the project management tools.

A significant relationship also exists between the organizational structure and risk. The utilization rate of stand-alone project structure was significantly higher for high-risk projects than for lower-risk projects. It is interesting to note that no relationship was found to exist between risk level and project size. Large projects were not found to be riskier than small projects.

Finally, project goals understanding, communication, problem handling, and team support were found to be negatively related to risk. As project risk increases, clearly establishing project goals, communication, and handling problems becomes more difficult. In order words, project risk is an impediment to communication and problem handling.

These findings show that the project management approach is related to the level of risk involved. In other words, risk influences the manner by which projects are managed. Also, risk amplifies the difficulties of project management. In the next section, it will be shown that the above management factors, in combination with risk, significantly influence success. As a consequence, these factors should carefully be considered from a strategic and tactical point of view, in combination with risk, while implementing projects.


Details of Findings

To test the hypothesis that the project management approach, in combination with risk, influences success, a regression model incorporating interactive terms was used (see Appendix for a description of the model). The model was used to test the influence of each of the three areas of risk considered: technical, cost, schedule risk. The results are given in the next three sections.

The Influence of Technical Risk. When technical risk is high, project success is significantly influenced by the level of authority assigned to the project manager, communication, team support, and problem handling (see Table A4 of Appendix). Again these results emphasize the importance of human relations in determining the likelihood of project success. In fact, all the more so when technical risk is high.

It was also observed that when technical risk is high, stand-alone project structure has a significantly negative influence on technical and cost success. A possible explanation is that technical expertise can more easily be drawn from a matrix structure than from a stand-alone project.

These results show that the higher technical risk gets, the more important it is to have team support, effective authority assigned to the project manager, effective problems handling, and effective communication. As well, stand-alone project structure should be avoided when technical risk is high.

These findings also indicate that PERT/CPM techniques have a significantly positive influence on time success when technical risk is high. Furthermore, monitoring has a significant influence on cost and overall success. These results show that high-technical-risk projects should be more carefully planned and more closely monitored to increase the likelihood of success.

The Influence of Cost Risk. Again, it was found that when cost risk is high, project success is significantly influenced by project goals understanding, the authority given to the project manager, team support, and communication (see data in Table A5 of Appendix).

It is interesting to note that when cost risk is high, formal project management tools, including the Work Breakdown Structure, C/SCSC, PERT/CPM, monitoring, and progress reports, do have a significant influence on project success. Riskier projects with regard to the budget require more formal planning, monitoring, and control tools.

Formal planning, monitoring, and control tools have significantly more influence on project success for high-risk projects than is the case for lower-risk projects. As a consequence, it is important to measure risk before deciding the most appropriate project management tools.

The Influence of Schedule Risk.

When schedule risk is high, two factors were found to significantly influence project success: the project manager's experience and the frequency of monitoring (see Table A6 of Appendix). The results of the study show that when schedule risk is high, experienced project managers achieve better project success than is the case with less experienced project managers. However, for lower-risk projects no significant difference in performance was observed between experienced and less experienced project managers.

Finally, the results indicated that high-schedule-risk projects should be more frequently monitored to increase the likelihood of success. Monitoring projects on a more frequent basis allows the manager to get more accurate data on the project status and to detect problems at an earlier stage. It is then easier to solve these problems and to stay on schedule.


Conclusion

In this research we showed that the best project management approach varies according to the level of risk involved in a project. The importance of project goal understanding, the level of authority given to the project manager, problem handling by the project team, communication, and team support was clearly demonstrated. While many success factors center around human relationship, this study also indicated that high-risk projects should be more carefully planned, closely monitored and controlled.

The following general recommendations are made (see Figure 1 for a decision flow chart):

  1. When technical risk is high:

    • Emphasize team support
    • Increase project manager authority
    • Improve problem handling and communication
    • Avoid stand-alone project structure
    • Increase the frequency of project monitoring
    • Use WBS, PERT/CPM, and C/SCSC techniques

  2. When cost risk is high:

    • Increase the frequency of project monitoring
    • Increase the frequency of project monitoring
    • Use WBS, PERT/CPM, and C/SCSC techniques
    • Improve communication, project goals understanding, and team support
    • Increase project manager authority

  3. When schedule risk is high:

    • Increase the frequency of project monitoring
    • Select the most experienced project manager

Project risk should be considered from a strategic and tactical point of view when implementing a project. Project success is significantly influenced by the selected management approach.


References

  1. Allen, T.J., D.M.S. Lee and M.L. Tushman. 1980. R&D Performance as a Function of Internal Commu- nication, Project Management, and the Nature of the Work. IEEE Transactions on Engineering Management, vol. EM-27, no. I (Feb.), pp. 2-12.

  2. Anderson, J., and R. Narasimhan. 1979. Assess- ing Project lmplementation Risk: A Methodological Approach. Management Science, vol. 25, no. 6 (Jun.), pp. 5]2-521.

  3. Aram, J.D., and S. Javian. 1973. Correlates of Success on Customer-lnitiated R&D Projects. IEEE Transactions on Engineering Management, vol. EM-20, no. 4 (Nov.), pp. 108-113.

  4. Dane, C.W., C.F. Gray and B.M. Woodworth. 1979. Factors Affecting the Successful Application of PERT/CPM Systems in a Government Organization. Interfaces, vol. 9, no. 5 (Nov.), pp. 94-98.

  5. Hughes, M.NX 1986. Why Projects Fail: The Effects of Ignoring the Obvious. E (Apr.), pp.l4-17.

  6. Kezsbom, D.S., D.L. Schilling and K.A. Edward. 1989. Dynamic Project Management, A Practical Guide for Managers ek Engineers. New York: Wiley.

  7. McFarlan, F.E 1981. Portfolio Approach to Information Systems. Harvard Business Review (Sep.-Oct.), pp. 142-150.

  8. McCollum, J.K., and J.D. Sherman. 1991. The Effect of Matrix Organization Size and Number of Project Assignments on Performance. IEEE Transac- tions on Engineering Management, vol. 38, no. 1 (Feb.), pp. 75-78.

  9. Might, R.J., and NZA. Fisher. 1985. The Role of Structural Factors in Determining Project Management Success. IEEE Transactions on Engineenng Manage- ment, vol. EM-32, no. 2 (May), pp. 71-77.

  10. Might, R. 1984. An Evaluation of the Effectiveness of Project Control Systems. EEE Transactions on EngineenngManagement, vol. EM-31, no. 3 (Aug.), pp. 127-137.

  11. Neter, J., and V. Wasserman. 1974. Applied Linear Statistical Models, p. 842. Illinois: Irwin.

  12. Pinto, J.K, and D.P. Slevin. 1988. Critical Success Factors Across the Project Life Cycle. Project Management Journal, vol. XIX, no. 3 (lun.), pp. 67-75.

  13. Pinto, J.K., and S.J. Mantel, Jr. 1990. The Cause of Project Failure. IEEE Transactions on Engineering Management, vol. 37, no. 4 (Nov.), pp. 269-276.

  14. Pinto, J.K., and D.P. Slevin. Critical Factors in Successful Project Implementation. 1987. EEE Transactions on Engineenng Management, vol. EM-34, pp. 22-27.

  15. Pinto, J.K., and D.P. Slevin. Project Success: Definitions and Measurement Techniques. 1988. Project Management Journal, vol. XIX, pp. 67-71.

  16. Rubin, I.M., and W Seelig. 1967. Experience as a Factor in the Selection and Performance of Project Managers. IEEE Transactions on Engineering Management, vol. EM-14, no. 3 (Sep.), pp. 131-135.


Appendix

Research Settings and Method

There were four steps in this research: selection of variables, questionnaire development, data collection, and statistical analysis of the data. These four steps are described in the following sections.

  1. Selection of Variables.

    The variables used in the research were selected via a literature review and interview with project managers. The variables were then divided into three categories: project success measures, project characteristics, and project management factors. The next step was to gather information about these variables.

  2. Questionnaire Development.

    Because the time allocated for the research did not allow for the personal contact of all the respondents, a questionnaire was used. The questionnaire was pre-tested by five project managers. According to the comments made by the reviewers, some questions were modified to facilitate reading. Over- all, the questions were deemed easy to understand and to answer.

    A seven-point Likert scale was used to assess the variables. An example of a typical question is given below.

    For this project, the support received from senior management is:

    Low ---- high 1 2 3 4 5 6 7

    For multiple-item scales, item-remainder coefficient, and coefficient alpha were calculated using SPSS/PC+ reliability analysis scale procedure. The item-remainder coefficient measures how well an item relates to the other items of a scale. Coefficient alpha reflects internal-consis tency reliability. All items selected for the study have an item-remainder coefficient of at least .40. For the selected scales, coefficient alpha varies between .78 and .96. It is generally accepted that coefficient alpha should be at least .70 for a scale to have internal consistency.

    The correlation within each of the three groups of variables was measured. The results, given in Eble A1, showed that they are truly measures of different phenomena. (See Table A1.)

  3. Data Collection. The questionnaire was sent to 163 project managers and project directors of 83 nearly or recently completed projects. Of 140 questionnaires returned, 121 valid questionnaires, corresponding to 73 projects, were obtained and used in this research.

    Of the 121 people who completed the questionnaire, 54 (44.6 percent) were project directors; 65 (53.7 percent) project managers; and 2 (1.6 percent) acted as both project manager and project director.

    The values (in Canadian dollars) of the projects in the sample are given in Table A2. The projects were on average 60 percent implemented (in dollar value). (See Table A2.)

  4. Statistical Analysis. To test the relationship between project management approach and success, regression analysis was used. The six measures of success successively are the dependent variables, and the 17 management factors are the independent variables (this is a total of 102 regressions). In this model, risk is not explicitly considered. The results of the regression analysis are summarized in Table A3. The table gives the value of the regression coefficient together with the level of significance (p-value). Only regression coefficients with a p-value less than .10 were reported. (See Table A3.)

    The relationship between risk, project management approach and success was then tested in two separate steps. First, the relationship between risk and the project management approach was determined using simple regression analysis. The 17 management factors were individually the dependent variables and the three risk variables were the independent variables (this is a total of 51 regressions). The regressions gave the relationship between risk and project management approach. The value of the regression coefficient and their p-value are given in Table A4. Only regression coefficients with a p-value less than .10 were reported.

    Secondly, the hypothesis that the interaction between the project management approach and risk significantly influence project success was tested using a multiple regression model employing