THE WELCH COMPANY
My interest is to arrest information entropy and hence provide real cost and schedule control through better management. Since management is 90% communication that is what needs to be improved.
A thesis that might evolve is whether a new concept of Communication Metrics can accomplish this aim, i.e., adding proactive controls to communication in order to make cost and schedule control more useful.
A 970328 report by the U.S. Army Corps of Engineers (USACE) finds that adding "metrics" to communication provides business "intelligence" that is missing from conventional computer solutions like wordprovessing, document management and email. This "intelligence" integrates time and information to link cause and effect for better decision support using a concept called "knowledge space." It requires a new kind of worker. Someone with the management experience and skill with specialized tools, plus the discipline to develop daily intelligence that makes the rest of the team more effective by avoiding information entropy. Thus, is it time to expand the "intelligence" function.
New workers to improve productivity are not new. Accountants were once new workers, as were architects, gas station attendents, and more recently cost engineers, software engineers and project managers.
The Communication Manager role is distinguished from a Project or Program manager in that the former conducts the meetings and calls and issues the decisions that generate communication. The Communication Manager (Project Administrator, Leadership Aide, Program Analyst -- the name is unimportant) adds "metrics" to communication to ensure understanding and follow up which others do not have enough time nor the psychological will to perform. The missing ingredient today that causes information entropy is the lack of intellignece that builds and maintains shared meaning, and follows up. So we assign someone to do that using special tools that make it possible.
Please let me know what you think. The first few pages of the USACE report are below the AFIT article.
Sincerely,
THE WELCH COMPANY
Rod Welch
Enclosures
Project Management Quarterly Vol 6 No. 3, 1975
A Decade of Project Management Project Management Institute Reprint copyright 1981 page 388
Uncertainty Analysis for Program Management
Martin Dean Martin Air Force Institute of Technology
John O. Lenz Air Force Systems Command
William L. Glover Wright-Patterson AFB
A major problem facing program managers responsible for Department of Defense (DOD) weapon systems development programs today is how to effec- tively predict and ultimately control and manage program cost growths. During the past ten to fifteen years, cost growths have plagued major development programs. There are many tools available for managers to use in estimating program costs, but most of the methods used do not consider the uncertainty associated with the successful completion of' a program in any rigorous or formal manner.
The specific inclusion of uncertainty in estimating program costs formed the basis of a recent research effort by the authors while assigned to the Air Force Institute of Technology (AFIT), Wright-Patterson Air Force Base, Ohio.'
Purpose of the Research
The purpose of the research effort was to validate an entropic cost model for use in predicting and controlling the final cost of weapon system development programs in the DOD. The model was originally formulated under Air Force sponsorship at the University of Oklahoma. A brief background will facilitate an understanding of the model.
To understand the entropic cost model, a few points must be introduced relative to the acquisition environment and the characteristics of information.
The Acquisition Environment
There are two phases of a development program which relate to the potential of a cost growth: pre-award and post-award (See Figure 1). For the entropic cost model, the critical point in time is the contract award for a development program.
Requirements Contract Contract
Generation Award Retirement
| | |
| | |
| | |
| Pre-Award | Post-Award |
| | |
| | |
+--------------------------+---------------------------+
Figure 1
Contract Life Cycle
During the pre-award phase (before actual award of the contract) the program
manager is primarily concerned with influencing future cost growth of his
program. To accomplish this task the program manager (PM) has at his disposal
certain information which should permit him to structure his decisions in a
rigorous manner at the time of contract award. This information includes
technical data, cost estimates, and results of risk analysis. Technical data
consists of engineering estimates and feasibility studies conducted by either
the government or the contractor. Cost estimates are available in four
principal forms: Cost Analysis Improvement Group (CAIG) estimates;
Independent Cost Estimates (ICE); estimates made by personnel organic to the
Systems Program Office (SPO); and finally, estimates in contractor proposals.
Uncertainty analysis is relatively new as far as being a formal and integral
part of the PM's information base; that is, only recently have serious
efforts been made to formalize and structure the process of uncertainty
analysis during the development program pre- award phase.
During the post-award phase (See Figure 1), the PM must monitor control systems and act to preclude a program cost growth based on his own expertise and that of his subordinates in the SPO. The information available to the PM has certain characteristics that are central to the fundamental concepts of the entropic cost model.
Characteristics of Information
The universe of program information relative to the development of a weapon system is comprised of two subsets: ordered information and information that lacks order. Ordered information relates to factors of the program which appear relatively certain as to their ultimate outcome (See Figure 2). These factors generally form the basis for the target (the theoretically "most likely") cost of' the program at contract award. The information in a program which lacks order relates to aspects of a program with uncertain outcomes and form the basis for cost growth during development and possibly production. A conceptual visualization of the SPO information base is illustrated in Figure 2.
Program Information Universe
xx
x
Order x
(Certainty) x
x Lack of Order
Basis for target cost x (Uncertainty)
x
x Cause of Cost Growth
x x
Figure 2
Program Information Base
The lack of order, or uncertainty, in program information forms the
foundation for the entropic cost model. In the terminology of set theory, it
is the complement of the ordered portion of the information base. For the
entropic model, the uncertainty of information is conceptualized as
approximately equal to entropy, or the disorder, in the program system of the
PM and his information. The concept of entropy has its roots in
thermodynamics. Entropy in a physical system is the amount of disorder
present in the system due to molecular state changes when heat is applied.
This property of disorder was extended to information systems in the
development of information and com- munication theory by Shannon and Weaver.
The concept was used in an attempt to explain noise in communication
systems.2 An extension of the concept forms the basis for the entropic cost
model.
An Entropic Cost Model
The entropic cost model is formulated as follows:
Final Program Cost = Target Cost
-----------------
Order in Information
Target Cost
= -----------------
1 - Disorder in Intormation
Target Cost
= -----------------
1 - Entropy
In this model, when order is dominant, the final cost is known. When order is
not present, there are multiple program factor outcomes possible, and the
final cost is uncertain. Thus, uncertainty implies several possible cost
outcomes. The purpose of the model is to estimate final cost by using a
quantified expression of the uncertainty in the program. The basic goal of
the recent research effort at the Air Force Institute of Technology was to
test the validity of the model, using data from an actual development
program.
Research Method
As a consequence of time constraints, a test using several development programs was not possible. Therefore, a single program was chosen as the test medium. The program ultimately selected was the Short Range Attack Missile (SRAM) development. Selection of the SRAM was predicated on the following factors:
(1) The program was recently completed;
(2) In terms of dollars, it was within the scope of
the effort;
(3) Personnel involved in the source contractor
selection for the SRAM were still at Wright-
Patterson AFB.
Validation of the model using the SRAM required a re-creation of the program
information environment in existence immediately subsequent to the award of
the development contract. A review of program documentation disclosed that
the data necessary to reconstruct the environment was contained in the
source-selection documentation file. This file was not readily available as a
result of the document security required by the source-selection process. An
alternative method was required to reconstruct the information base for the
test of the entropic cost model.
Reconstruction of the Post-Award Information Environment
The limited access to documentation caused an additional secondary objective to be added to the research. That was, in essence, to develop a method by which the uncertainty of information could he quantified in a structured manner. The method needed to not only satisfy accepted research tech- niques, but be practically useful, if possible, in the realm of the PM and his decisions.
The work of C. Jackson Grayson in his organization of expert judgment and its application to oil well drilling decisions was known to the researchers. The technique rested largely on the use of probability statements as responses to questions of well-drilling experts relative to the potential success of the drilling operation in terms of oil production. Specifically, Grayson requested geologist faced with making decisions relative to the location of oil-bearing formations to develop probability distributions as to the probable success of drilling at a specific location. The geologist could, then, subjectively formulate a risk function based on his experience or could select a "Classical" distribution, such as Normal, Gamma, or Poisson, which seemed to fit his mental pattern. The individual's subjective risk function (distribution) would be derived from a verbal lottery with successive questions to specify quantitative point estimates, which when plotted would give a probability distribu- tion. Later, Grayson extended this technique to derive a group risk preference function. A similar type of question/answer format was perceived to be applicable to the program management environment in DOD.
A somewhat exhaustive review of techniques used to structure the opinions of experts revealed the DELPHI method, developed at the RAND Corporation, as a candidate for application of the Grayson method to the instant research. DELPHI is a method for predic- ting the probability of future events by polling experts, concerning their subjective evaluation as to event occurrence. Each participant is interrogated individually by means of an interview or questionnaire. The process involves four rounds of interrogation. The results of each round are fed back to the participants in an anonymous manner to eliminate the influence of strong personalities. The goal is to refine and revise the subjective probabilities which are being formulated as a measure of the uncertainty of occurrence for each future, alternative outcome. Generally, the DELPHI technique is normally used in forecasting from the present to the future, with responses in the form of what might happen. As applied to the recollection of SRAM program source-selection panel members, the responses not only expressed what was to happen, but assigned probability statements to the outcome measures of unacceptable, acceptable, and exceptional.
By the controlled interview/feedback/interview cycle central to the DELPHI method, the researchers were able to identilv some 19,683 possible SRAM program factor-outcome combinations, and assign a probability to each. The calculation base was the nine (9) factors identified during the DELPHI interroga- tion; each having three categories of outcomes assigned. Thus, there were 39 or 19,683 possible outcomes or states for the program. By means of the computer, entropy was then calculated using the following formula:
19,683
ä Pi log Pi
i = 1
Entropy = System Entropy = -----------------
Maximum Entropy log 19.683
= 0.686
Where, 0 <= entropy <= 1 and
Pi = probablity of the ith program factor
outcome combination
The entropic value of 0.686 was used to compute an estimate for the total
program cost as outlined below:
Total Program Cost = Target Cost
1 -Entropy
= $143.3 Million (M)
1 - 0.686
= $143.3 M
0.314
= $456.4 M
Subjective Probabilities by Means of DELPHI
The DELPHI method was used to poll the original participants in the SRAM source selection to determine the subjective probabilities associated with the uncertain outcomes expected in the development program for the SRAM. These probabilities were used to calculate the entropy in the program at the time of source selection on a retrospective basis. As related to the validation goal for the model, the findings for the effort are significant.
Findings and Conclusions
The actual total cost for the SRAM development program was $439 million. The estimate for this cost obtained by applying the entropic cost model was $456 million. This estimate was based on encountering the worst possible cost conditions during development. Adjustments based on approved changes which were not contemplated at the time of source selection were made to the final cost data. As a consequence, results of the study indicate that the entropic cost model is a valid predictor of development program cost. The power of the model rests in its ability to readily explain uncertainty in a single measure, entropy. Admittedly,, the results of' one research effort do not validate the model for general applicability. However. the model does have potential as a cost estimating tool for program managers. Further research applying the model to other developmental programs to determine the extent of the usefulness of the entropy concept is planned.
Another significant finding that was a by-product of the research endeavor was the use of DELPHI in uncertainty analysis. Application of the DELPHI methodology to determine uncertain aspects of' a development program provides a structured process by which a PM can use his experts to develop rigorous inputs to assist in making key and significant program decisions as related to cost, time, and performance.
Other Possible Applications
Both the DELPHI technique and the entropic cost model merit consideration for application in various areas external to the defense environment. Planning and control for a large, high-dollar value project which entails a moderate to high degree of uncertainty as related to cost could be managed by application of'the methodology at selected decision points over time.
For example, large-scale projects in construction, marketing new products or services, advancing technology and exploration for new deposits of natural resources are a few possible areas for application. The amount of effort expended naturally depends on the magnitude of the project and how much time and money management is willing to invest for informa- tion, a measure of the entropy in the information base, and the estimated cost outcome for a specific program. Certainly, the benefits as related to cost control should exceed the cost of administration.
FOOTNOTES
1. William L. Glover and John O. Lenz, A Cost Growth Model For Weapon System Development Programs, Unpublished Master's Thesis (Wright- Patterson Air Force Base, Ohio: Air Force Institute of Technology, 1974), p. 124.
2. Claude E. Shannon and Warren Weaver, The Mathematical Theory of Communication (Urbana, Illinois: The University of Illinois Press, 1949). p. 23 .
3. C. Jackson Grayson, Jr., Decisions Under Uncer- tainty (Boston, Massachusetts: Harvard Business School, Division of Research, 196O, p. 70.
BIBLIOGRAPHY
1. Glover. William L. and Lenz,.John O., A Cost Growth Model for Weapon System Development Programs. l published Master's Thesis. Wright-Patterson Air Force Base. Ohio: Air Force Institute of Technology, 1974.
2. "Shannon, Claude E. and Weaver, Warren, The Mathemati›ai TheorN of Communication, Urbana. Illinois: The University ot' Illinois Press, 1949.
3. Grayson, C..Jackson. [??], Decisions under Uncertainty, Boston, Massachusetts: Harvard Business School. Division of Research, 196[?]
U.S. Army Corps of Engineers San Francisco District 333 Market Street San Francisco, CA 94105 1905
March 28, 1997
Corps of Engineers
COMMUNICATION METRICS
The Welch Company, Contract DACW07-96-P-0705
Submitted to:
Thomas Benero Chief, Contracting Division 415 977 8511
Prepared by:
Thompson F. Keesling Assistant Chief Construction Operations Division 415 977 8701
CONTENTS
1. Executive Summary
Intelligence Converts Information into Knowledge ....................... 1
2. Communication Manager: New Role, Tools and Skills ..................... 1
3. Background - Oakland Project Increased Communication Needs ............. 3
4. Hiring Welch - Communication Metrics, a New Management Method .......... 5
5. Welch Contract Performance - "Paperless Office" a Reality? ............. 8
6. Evaluation - Technology Improves Leadership and Management ............. 9
7. Conclusion - Wider Use of Communications Metrics Warranted ............. 11
Comments by COE Managers .......................................... Appendix A
ax Blodgett, Chief, CON OPS, (415 977 8444)
eonard SooHoo, Chief, Constr. Srvcs. Branch, (415 331 0404)
erb Cheong, Project Manager, PPMD, (415 977 8705)
arc McGovern, Construction Manager, CON OPS, (415 977 8467)
SDS Evaluation, COE Letter Feb 19, 1997............................ Appendix B
PG&E Report Dec 30, 1994........................................... Appendix C
Scope of Services for Communication Metrics........................ Appendix D
Subject Index on Richmond Project Schedule/WBS..................... Appendix E
1. Executive Summary
Intelligence Converts Information into Knowledge
The Welch contract required performance of "Communication Metrics" on the
Oakland Harbor project. This is a new Risk Management method of decision
support that uses technology to track the connections in human thought from
integrating "time" and "information." Lt. General (Ret) Henry J. Hatch, former
Commander of the Corps of Engineers, describes Communication Metrics as a
business "Intelligence" role to organize and analyze information, so that cause
and effect, and needed actions are revealed to make leadership effective. We
found Communication Metrics improved our results. Since it is a new business
process for managing more carefully, this report provides a basis for future use.
Welch explains Communication Metrics derives from cognitive science theory on converting "information" into "knowledge." Time produces patterns (i.e., organic structures) of information that link cause and effect. Analysis discovers cause and effect by organizing and summarizing details of daily experience. This "intelligence" is vital to leadership because management is mostly communication from dialog and documents. Patterns of cause and effect are obscured by high information flows. When there is not enough time for analysis, understanding fragments. People trying to "expedite" get mixed up without realizing it. Decisions become disconnected from relevant details, causing a morass of mistakes under the rule that "Haste makes waste!" Adding "metrics" to communication is a Risk Management solution. Welch describes a concept of "knowledge space" that integrates time and information. Critical details are summarized to facilitate understanding and maintain shared meaning among team members. Follow-up actions are linked to original sources, so that decisions are aligned with objectives. Empowering people to use business "intelligence" for daily work leverages the innate mental process of converting information into knowledge, which is the essence of "managing." Apart from theoretical explanations, the San Francisco District found that Communication Metrics improves teamwork, productivity and cost savings. We conclude it has potential for wider use by the Corps of Engineers (see Appendix A). Scope of services to use this method on new projects is in Appendix D. Application to support the schedule and WBS for our Richmond project is in Appendix E.
2. Communication Manager: New Role, Tools and Skills
Rod Welch developed the technology called the Schedule Diary System (SDS) and methods he calls "Communication Metrics," as a new management science. He performed the work under this contract using SDS to schedule tasks linked to an electronic diary (hence, "Schedule Diary System"). The diary is a data- base of daily dialog and documents that together comprise "communication." Rod argues today's "Information Highway" of constant meetings, calls, email and reports overwhelms people and frustrates leadership. There is not enough time using conventional methods for people to understand and follow-up. Limited time means more resources are needed. "Intelligence" is one way of investing resources to support leadership. Government agencies provide military "intelligence" (e.g., CIA). But, it is expensive, and so is used only at the highest levels. Communication Metrics applies this strategy more broadly by using SDS technology to capture, organize, verify, analyze and link the daily record. A continual process of investing intellectual capital brings innovation, knowledge and ideas. Technology and special skills perform this work in less time to make business "intelligence" an economically viable new role.
Communication Metrics adds value to our "Lessons Learned" process because the SDS schedule is used to manage the work each day based on linkages to the diary and contract requirements. We found it takes experience to learn what this means; but, it seems to improve performance, because relevant parts of the record, over days, weeks or years, can be assembled instantly. Faster command and control of the record enables reliance on accurate information. Linkages provide consistent understandings of communications. Analysis improves traditional management by hunch and conversation (i.e., "guess and gossip.") Follow-up improves because reporting and analysis (writing and linking) on what was done reveal new tasks that need to be done which are scheduled and automatically linked back to the source record. Synergy from added value due to faster access and frequent use encourages managers to create better records. Better understanding and follow-up yield better results, better attitudes and team participation. The Corps of Engineers notified Welch in a letter dated February 19, 1997 that "Your expertise and application of the SDS program contributed to establishing order to a contract which was going out of control" (Appendix B). Deliverables were received on time or ahead of schedule and exceeded government standards for quality and impact. Rod calls the new job a "Communication Manager" who uses SDS skills to facilitate management in being prepared and following up. It applies General Hatch's point that "intelligence" adds value to information so leadership can be effective.
This report is longer than typical contract evaluations because it covers a new methodology that turned out to vary from original expectations. We have previously described it as a new form of "documentation" because that term is familiar to commanders and executives who use information. Writing in SDS, however, is aimed at discovering the correlation of events that make up human understanding and testing it for accuracy (i.e., alignment), thus "metrics;" and, then using the resulting record as a constantly expanding resource to measure future understandings in the way "experience" is used by the mind. It supports the management cycle of "plan, perform, report" to produce "intelligence," so that managers are prepared to perform their work. Current trends of increased information flows from meetings, fax, calls and email, require constant search and assessment by the Corps of Engineers for better tools and methods. We report some background on the Oakland Harbor project, the process of hiring Welch, and details on work performance in order to develop a working definition of Communication Metrics, based on how it managed increased information flows on a difficult project.