THE WELCH COMPANY
440 Davis Court #1602
San Francisco, CA 94111-2496
415 781 5700
rodwelch@pacbell.net


S U M M A R Y


DIARY: December 13, 1999 10:05 AM Monday; Rod Welch

NSF responds to proposal on Com Metrics.

1...Summary/Objective
2...NSF Responds to Proposal for Communication Metrics Management Science
3...Computer Science Review Panel May Be Incorrect for Com Metrics
4...Com Metrics Needs Review by Management, Cognitive, Computer Science
5...NSF Found Communication Metrics Sets Important Goal
6...Click Links Omitted Ignorance SDS Exists Seems Like Only Theory
7...NSF Concluded Com Metrics is Only Theory, Due to Inexperience
8...NSF Finds Need for "Intelligence" Not Established by Proposal
9...Knowledge Management Delimma Undefined by NSF
10...NSF Found Research Plan Defective, Principal Investigator Lacks Skill
.....4...Computational Linguistics combines humanities, natural and
11...NSF Practice Justifies Support for Communication Metrics
12...Appeal Procedures for NSF Grant Determinations


..............
Click here to comment!

CONTACTS 
0201 - National Science Foundation                                                                                                                                        O-00000716 0503
020101 - Ms. Sarah Nerlove; 703 306 1395 Ext 5247                                                                                                                         O-00000716 0503

SUBJECTS
NSF Proposal 9961176
Administration, Receipt, Processing
Award Schedule
Atititude Makes Resistance to Communication Metrics Emotional

1006 -
1006 -    ..
1007 - Summary/Objective
1008 -
100801 - Follow up ref SDS 46 0000, ref SDS 42 0000.
100802 -
100803 - NSF found Communication Metrics is a good idea. ref SDS 0 9435
100804 - Reviewers suggest the "need" for adding "intelligence" to management,
100805 - and research planning can be improved.  There is no comment on the
100806 - underlying goal to strengthen alphabet technology, i.e., literacy, and
100807 - no reference to "intelligence" cited in the proposal as foundational
100808 - to using technology usefully for management.  No other awards are
100809 - cited for a system of Communication Metrics, so duplication does not
100810 - seem to have been an issue.  It appears Reviewers did not access the
100811 - proposal on the Internet, as recommended by NSF staff to demonstrate
100812 - Com Metrics has been implemented and found to deliver useful
100813 - "intelligence." ref SDS 0 1764  This omission colored the review with
100814 - a false impression of exaggerated theory, ref SDS 0 2R9G, and
100815 - inability to perform due to lack of credentials, that resulted in not
100816 - making an award, because reviewers lack experience in the field of
100817 - Knowledge Management, even though they may be widely read on the
100818 - subject. ref SDS 0 0865,
100820 -  ..
100821 - Another factor may have been review by computer science, rather than
100822 - inclusion of management, behavioral science expertise on review panel.
100823 -
100824 -     [On 991221 called Sally at NSF on review. ref SDS 54 0001]
100825 -
100826 -     [On 000124 notified Roy Roebuck on results. ref SDS 57 0001]
100828 -      ..
100829 -     [On 000510 Doug Engelbart was denied NSF support on grounds that
100830 -     "knowledge" was not defined in proposal. ref SDS 62 0004
100831 -
100832 -
100834 -  ..
1009 -
1010 -
1011 - Progress
101201 -  ..
101202 - NSF Responds to Proposal for Communication Metrics Management Science
101203 -
101204 - Received ref DRT 1 0001 signed by the Division Director, Louis A.
101205 - Martin-Vega, responding to proposal 9961176 for Com Metrics, submitted
101206 - on 990616. ref SDS 27 0001, via transmittal letter, ref DIP 3 0977
101207 - which included the....
101208 -
101210 -                    ..
101211 -                   Project Summary, ref OF 11 0001
101212 -                   Table of Contents, ref OF 13 4144
101213 -
101214 -
101215 - ...based on objectives set on 990225. ref SDS 18 0788
101217 -  ..
101218 - NSF does not reference receipt on 990827 of ref DIP 14 6399 to Sally
101219 - proposing to change commercial focus of the project to Common
101220 - Administration. ref SDS 46 0001  Lack of critical analysis in the
101221 - review, indicates this added information may not have influenced the
101222 - result.
101224 -  ..
101225 - NSF missison is reported on 990404. ref SDS 21 2112
101227 -  ..
101228 - NSF cannot support the proposal for Communication Metrics, ref DRT 1
101229 - 0001, based on reports by the review panal, limited funding,
101230 - duplication and NSF program balance. ref DRT 1 2365
101231 -
101232 -     NSF web site shows no awards that duplicate the scope of the
101233 -     proposal for Com Metrics.
101235 -      ..
101236 -     [On 991221 called and talked to Sally who advised awards for the
101237 -     current peiord will not be posted to the web until next month; she
101238 -     did not cite any other projects that support Communication
101239 -     Metrics. ref SDS 54 0001]
101241 -  ..
101242 - Inquires should go to Sally. ref DRT 1 4672
101244 -  ..
101245 - The second page has general information on NSF procedures. ref DRT 1
101246 - 5280  It says....
101247 -
101248 -     1.  Approximately 200 awards were made out of 1420 proposals.
101249 -         ref DRT 1 5544
101250 -
101251 -     2.  Reviews are advisory input to NSF. ref DRT 1 1974
101252 -
101253 -         Reviewers may cite specific references in the proposals and
101254 -         offer suggestions for improvements. ref DRT 1 1974
101256 -          ..
101257 -     3.  NSF may consider factors other than the reviews. ref DRT 1
101258 -         6240
101260 -          ..
101261 -     4.  Some meritorious proposals could not be approved due to lack
101262 -         of funds. ref DRT 1 1024
101264 -          ..
101265 -     5.  Award factors are specified again. ref DRT 1 1024
101266 -
101267 -          •  Potential commercial application,
101268 -          •  Past performance commercializing SBIR/STTR results
101269 -          •  NSF program emphasis and balance.
101270 -
101271 -
101273 -  ..
101274 - Computer Science Review Panel May Be Incorrect for Com Metrics
101275 - Com Metrics Needs Review by Management, Cognitive, Computer Science
101276 -
101277 - The NSF Review Panel for...
101278 -
101279 -
101280 -                   OOSBlIT-19 Intelligent Systems
101281 -
101282 -
101283 - ...considered the proposal on Com Metrics. ref DRT 1 0024
101285 -  ..
101286 - This met the goal of introducing new ideas for improving management
101287 - and earnings set on 990225. ref SDS 18 0788
101289 -  ..
101290 - Typically, "intelligent systems" work is focused in computer science,
101291 - for artificial intelligence that is theoretical, rather than practical
101292 - application, as seen from the review of Campbell's book on 900303,
101293 - ref SDS 1 4284, and work on "mediators" at Stanford, reviewed on
101294 - 940603. ref SDS 2 0650
101296 -  ..
101297 - Communication Metrics is a practical method of improving use of the
101298 - alphabet that auguments innate human capabilities to plan, perform and
101299 - report on the work. ref OF 1 1104  An example is this record capturing
101300 - the chronology of experience with connections of cause and effect that
101301 - are segmented into chunks of context relative to objectives. This body
101302 - of "intelligence," under the definition in POIMS, ref OF 1 0582, can
101303 - then be delivered via Internet to build and maintain shared meaning
101304 - for effective collaboration on common objectives, as set out in the
101305 - Communication Metrics web site...
101306 -
101308 -                       ..
101309 -                      http://www.welchco.com
101310 -
101312 -  ..
101313 - On 990715 Sally objected to this method, ref SDS 37 0001, so this may
101314 - be an influence in the review.
101316 -  ..
101317 - Like the alphabet, SDS is a tool that leverages human thinking,
101318 - explained in POIMS ref OF 1 0858 and ref OF 1 6528   The proposal
101319 - states this expressly in the first para, ref OF 14 0001  Like the
101320 - alphabet, SDS does not "think" nor seek to accomplish intelligence for
101321 - people, but rather helps people think for themselves and generate
101322 - useful "intelligence."  Therefore, reviewing Com Metrics that applies
101323 - SDS requires experts who have investigated the nature of thinking, as
101324 - in Campbell's book, and Landauer's paper on LSA reviewed for example
101325 - on 960518. ref SDS 7 8811
101327 -  ..
101328 - Proposal Part 2, Significance of Innovation cites reliance on
101329 - cognitive science in the first paragraph, and the objective to support
101330 - Enterprise Management. ref OF 14 0001
101332 -  ..
101333 - The NSF review panel may have had people with management and
101334 - behavorial science expertise, however, that expertise is not evident
101335 - in comments submitted by NSF.  Moreover, "Intelligent systems" often
101336 - connotes computer scientists.  People familiar with management
101337 - science, connectionist theory and alphabet technology need to assess
101338 - the proposal, as discussed with Cheryl and Joe on 990614, ref SDS 24
101339 - 0866, confirmed in the letter to NSF on 990615. ref DIP 2 0001
101340 -
101341 -     [On 991221 Sally Nerlove at NSF was not aware that Com Metrics
101342 -     needs cognitive science review. ref SDS 54 5014]
101344 -      ..
101345 -     [On 000510 Doug Engelbart was denied NSF support on grounds that
101346 -     "knowledge" was not defined in proposal. ref SDS 62 0004
101348 -  ..
101349 - If NSF has a program for management science, possibly further review
101350 - is warranted based on using technology to leverage human cognition.
101352 -  ..
101353 - The SDS program and the Internet use computers.  Such experts should
101354 - have something helpful to contribute, so long as the objectives of the
101355 - proposal are recognized in evaluating the proposal.  Here that appears
101356 - not to have occurred.
101357 -
101358 -
101359 -
101360 -
101361 -
1014 -

SUBJECTS
Panel Conclusions
Willfully Blind, Fear and Denial

1204 -
120501 -  ..
120502 - NSF Found Communication Metrics Sets Important Goal
120503 -
120504 - NSF Review requires...
120505 -
120506 -     State the proposal's aim; comment on the proposal's strengths,
120507 -     weaknesses and commercial potential as discussed by the panel;
120508 -     and, suggest improvements for the research plan and or proposal
120509 -     preparation. ref DRT 1 2548
120511 -  ..
120512 - NSF Review Panel found...
120513 -
120514 -     The researcher proposes to create an internet search capability
120515 -     that allows users to assign multiple subjects to chunks of
120516 -     information and to chain information by time. The claim is that
120517 -     this would facilitate information retrieval. ref DRT 1 1260
120519 -      ..
120520 -     This is an ambitious goal but the argument for the need should be
120521 -     more clearly articulated and the proposal needs a clearly
120522 -     formulated research plan. ref DRT 1 3871
120524 -  ..
120525 - The lead reviewer states...
120526 -
120527 -     I can imagine that a highly functional system of the kind hinted
120528 -     at in the proposal could make a substantial contribution to
120529 -     knowledge management and, in a period when that phrase is
120530 -     increasingly popular in data mining systems, electronic commerce,
120531 -     and similar applications areas, that it could offer commercial
120532 -     value. But there is no evidence that what is proposed here would
120533 -     be a highly functional system. ref DRT 1 3630
120535 -  ..
120536 - Second Reviewer says...
120537 -
120538 -     Retrieval from experience is certainly desirable. ref DRT 1 2244
120539 -
120541 -  ..
120542 - Third Reviewer says...
120543 -
120544 -     The PI proposes to improve enterprise management by developing
120545 -     business metrics for improved communication.
120546 -
120547 -     The goal is certainly ambitious.  However, the text is unclear as
120548 -     to what will be done, hugely overstated in its claims, and
120549 -     entirely lacking in any documentation of these claims. ref DRT 1
120550 -     3723
120551 -
120553 -  ..
120554 - Click Links Omitted Ignorance SDS Exists Seems Like Only Theory
120555 - NSF Concluded Com Metrics is Only Theory, Due to Inexperience
120556 -
120557 - On 990614 NSF staff Joe Hennessy and Cheryl Albus recommended that
120558 - the Welch proposal provide access to the SDS record on the Internet so
120559 - that reviewers can understand ideas on "intelligence," and recognize
120560 - that Communication Metrics is not just a theory, but rather is a
120561 - practical solution to an important problem, and has been largely
120562 - implemented. ref SDS 24 5451  NSF's recommendation applies Einstein's
120563 - observation that experience is the foundation of knowledge, reported
120564 - on 991124. ref SDS 51 3066
120566 -  ..
120567 - On 990615 Joe Hennessy reported having experienced Communication
120568 - Metrics on the Internet and confirmed the prior recommendation that
120569 - this is helpful to understand the proposal. ref SDS 25 1044
120571 -  ..
120572 - Joe and Cheryl also recommended that a list of potential reviewers who
120573 - have experience with the SDS record on the Internet be submitted to
120574 - NSF for selection of a review panel. ref SDS 24 9903
120576 -  ..
120577 - On 990616 a list of reviewers was submitted, and the proposal provides
120578 - direct access to the SDS record on the Internet.
120580 -  ..
120581 - NSF review comments show failure to review the Com Metrics proposal
120582 - online, as requested in the proposal. ref OF 14 4402  Using the
120583 - printed version prevented gaining experience showing implementation of
120584 - Com Metrics that enables a "paperless office" that augments human
120585 - intelligence, causing a false conclusion that Com Metrics is only
120586 - theory, and cannot be accomplished, further mirroring the error made
120587 - by peer review at Intel on 990713. ref SDS 34 1767
120588 -
120589 -     [On 020726 researcher left SRI because of slow progress on work
120590 -     sponsored by NSF and EPA. ref SDS 69 K16J
120592 -  ..
120593 - Lead Reviewer says...
120594 -
120595 -     "... can imagine that a highly functional system of the kind
120596 -     hinted at in the proposal could make a substantial contribution to
120597 -     knowledge management and, in a period when that phrase is
120598 -     increasingly popular in data mining systems, electronic commerce,
120599 -     and similar applications areas, that it could offer commercial
120600 -     value. But there is no evidence that what is proposed here would
120601 -     be a highly functional system," ref DRT 1 3630,
120603 -  ..
120604 - The reviewer errs describing the Welch proposal as "hinting" at a
120605 - highly functional system, since the proposal expressly states the
120606 - system lifts the capacity to think, remember and communicate, and it
120607 - provides direct evidence, not hints, that the system is functional.
120609 -  ..
120610 - Aligns with review by Intel on 990713 who were ignorant of SDS because
120611 - they were shielded from the record on the Internet. ref SDS 34 2150
120612 -
120613 -      [On 010425 Morris reported SDS use cases are a "utopia" compared
120614 -      to methods other people use, which reflects NSF review that Welch
120615 -      proposal is overstated, and cannot be performed. ref SDS 67 EP7F
120617 -  ..
120618 - Struggeling to "imagine" Com Metrics would be useful rather than
120619 - commenting on experience interacting with the SDS record on the
120620 - Internet, shows the Reviewer is unaware that the system already
120621 - exists, and is therefore fully functional.
120622 -
120623 -      [On 000503 Jack Park had similar experience with NIH. ref SDS 61
120624 -      3025
120626 -       ..
120627 -      [On 000510 Doug Engelbart was denied NSF support on grounds that
120628 -      "knowledge" was not defined in proposal. ref SDS 62 0004
120630 -  ..
120631 - Reviewer's contention there is no evidence that what is proposed would
120632 - be a highly functional system is incorrect, since the existence of the
120633 - SDS record on the Internet is direct evidence that Communication
120634 - Metrics is a functional system. Second, reports by the U.S. Army Corps
120635 - of Engineers that Communication Metrics was used to save money for
120636 - taxpayers is cited three (3) times in the proposal, ref OF 14 0556 and
120637 - ref OF 14 2124, and comprises direct evidence of a highly functional
120638 - system, since ROI is 10:1.
120640 -  ..
120641 - Second reviewer says...
120642 -
120643 -     "Retrieval from experience is certainly desirable. However, in
120644 -     this system what the user can retrieve is only as good as the
120645 -     information he inputs.  This is a major flaw as most users will
120646 -     not have time to do all this input." ref DRT 1 2244
120648 -  ..
120649 - Reviewer's comment shows failure to access the proposal online, since
120650 - doing so reveals a much wider range of context is provided by SDS than
120651 - merely what is "input" by a user, as shown by this record, which is
120652 - linked to the NSF review, NSF requirements, and other sources that
120653 - have not been input by the user, but bear on the meaning and action to
120654 - be taken by the user and others.  That is the magic of "intelligence"
120655 - that adds value to information by creating "knowledge," and is
120656 - overlooked today by NSF, because they do not know the distinction
120657 - between information and knowledge, and do not have the benefit of
120658 - using SDS, but are merely passing on whether it would be helpful to
120659 - have something about which they have chosen to be ignorant by refusing
120660 - to click on a link.
120662 -  ..
120663 - Refusal by NSF reviewers to access links in the proposal aligns with
120664 - complaints on 990715 by Sally Nerlove, NSF lead for this proposal
120665 - area, ref SDS 37 0001  It, also, aligns with cultural resistance to
120666 - new methods embodied by the legend of Prometheus reviewed on 991108.
120667 - ref SDS 50 5368  While resistance to new methods should be expected
120668 - (Andy Grove notes people like to work on familiar things in familiar
120669 - ways, reviewed on 980307. ref SDS 15 3740) some accomodation should be
120670 - expected by the agency entrusted to assess new methods.
120672 -  ..
120673 - Belief that retrieval in SDS is only as good as information "input"
120674 - ignores the value of connections human intelligence adds to otherwise
120675 - improvished information, well recognized in cognitive science, see for
120676 - example review on 990303 of George Miller's early work. ref SDS 19
120677 - 2838  Previous review on 900303 made the same point. ref SDS 1 4456
120678 - Experiencing SDS online shows that many connections human intelligence
120679 - makes are in SDS records.  So much so, that some people are
120680 - overwhelmed, as reported on 980712. ref SDS 16 6804  Sally may have
120681 - had this experience on 990715, ref SDS 37 0001, leading to the NSF's
120682 - unfavorable review today.
120684 -  ..
120685 - Moreover, experiencing "intelligence" on the Internet exposes the flaw
120686 - in the hunter/gatherer view that executives, lawyers, engineers,
120687 - doctors, mechanics and other knowledge workers, do not have time to
120688 - invest intellectual capital, since in fact reliance on accurate
120689 - experience is the only method humans have to "save time," as reviewed
120690 - on 950426. ref SDS 3 4404
120692 -  ..
120693 - The third review says...
120694 -
120695 -     "The goal [of business metrics for communication] is
120696 -     certainly ambitious. However, the text is unclear as to
120697 -     what will be done, hugely overstated in its claims, and
120698 -     entirely lacking in any documentation of these claims.
120699 -     ref DRT 1 3723
120701 -  ..
120702 - Concluding the proposal is unclear as to what will be done conflicts
120703 - with the first two reviews, and indicates cursory consideration, per
120704 - above, ref SDS 0 0940, as occurred at Intel on 990713, where three
120705 - managers overlooked the problem of Information Overload cited in the
120706 - first paragraph of the proposal. ref SDS 34 9900
120707 -
120708 -     [On 000716 Professor Joseph Ransdell reported Com Metrics is
120709 -     ambitious because it provides a theory of management that has
120710 -     eluded disocvery since the 17th century, and he urged not to get
120711 -     discouraged by slow acceptance. ref SDS 64 7838
120713 -  ..
120714 - The feeling that a claim of delivering "intelligence" on the Internet
120715 - is "overstated," shows failure to experience Communication Metrics on
120716 - the Internet.  The existence of the implementation shows the ability
120717 - to perform as noted by Joe Hennessy on 990614. ref SDS 25 1044
120719 -  ..
120720 - The implication is that very capable people are numbed and frozen by
120721 - the prospect of adding intelligence to management, and simply cease to
120722 - think rigorously and honestly when confronted with a proposal to
120723 - improve their work, as observed by Andy Grove in his book "Only the
120724 - Paranoid Survive," reviewed on 980307. ref SDS 15 2044
120725 -
120726 -
120728 -  ..
120729 - NSF Finds Need for "Intelligence" Not Established by Proposal
120730 -
120731 - NSF Review requires...
120732 -
120733 -     State the proposal's aim; comment on the proposal's strengths,
120734 -     weaknesses and commercial potential as discussed by the panel;
120735 -     and, suggest improvements for the research plan and or proposal
120736 -     preparation. ref DRT 1 2548
120738 -  ..
120739 - Panel found no systematic argument on the need for Com Metrics...
120740 -
120741 -     The researcher proposes to create an internet search capability
120742 -     that allows users to assign multiple subjects to chunks of
120743 -     information and to chain information by time. The claim is that
120744 -     this would facilitate information retrieval. While the researcher
120745 -     addresses the need for this in very general claims, there is no
120746 -     systematic argument for this need. ref DRT 1 1260
120748 -  ..
120749 - Lead Review says...
120750 -
120751 -     The proposal references a critical information management need -
120752 -     essentially, organizing time and information through software that
120753 -     supports intelligent, shared linkages. It does so without defining
120754 -     in any systematic way the nature of that need,... ref DRT 1 8700
120756 -  ..
120757 - Analysis...
120758 -
120759 -     1.  NSF Review does not align well with proposal objectives to
120760 -         deliver "intelligence" on the Internet.
120761 -             ..
120762 -         a.  The project summary says...
120763 -
120764 -             Research objectives are (1) to create a search capability
120765 -             on the Internet that assembles chains of chronology by
120766 -             subject content to show causation over specified time
120767 -             spans; (2) a means is needed to assign multiple subjects
120768 -             to chunks of information based on personal and
120769 -             organizational objectives that control how information is
120770 -             assembled for discovering causation, and to share
120771 -             knowledge structures that build and maintain shared
120772 -             meaning; (3) guidelines for clear, concise, complete
120773 -             communication support effective use of the Internet.
120774 -             ref OF 11 6800
120776 -              ..
120777 -         b.  Significance of Innovation says...
120778 -
120779 -             The project will research Communication Metrics on the
120780 -             Internet to deliver intelligence ... the human brain
120781 -             connects information into chains of cause and effect
120782 -             through a process called "intelligence." This new project
120783 -             proposes research on using technology that leverages
120784 -             innate intelligence by integrating time and information,
120785 -             which produces "knowledge,"  This adds value to human
120786 -             experience... [supports] paperless office. ref OF 14 0001
120788 -              ..
120789 -             Another major innovation in SDS are tools and procedures
120790 -             for organizing information according to an organic subject
120791 -             structure to facilitate retrieval. ref OF 14 3922
120793 -  ..
120794 - Accordingly, NSF review can be strengthened by offering a richer
120795 - explanation based on the language in the proposal.
120797 -  ..
120798 - An example to improve NSF review is as follows...
120799 -
120800 -       The Welch proposal claims SDS technology facilitates connecting
120801 -       information into chronologies of cause and effect segmented by
120802 -       subject content, as an extension of alphabet technology, for
120803 -       capturing and managing daily experience in relation to personal
120804 -       and organizational objectives.  The methodology is summarized by
120805 -       the notion of integrating time and information.  Documents and
120806 -       Contacts are integrated into the experiential record for context
120807 -       management.  Chronologies of cause and effect are claimed to
120808 -       provide traceability to original sources.  This method reflects
120809 -       root cause analysis, case studies, lessons learned, history, or
120810 -       more simply what is commonly called "knowledge."
120812 -        ..
120813 -       "Intelligence" is defined in the proposal as a continual process
120814 -       of plan, perform, report to organize, align, analyse, develop
120815 -       summary connected to detail, provide and obtain feedback.
120816 -       Alignment identifies deviations from facts, objectives and
120817 -       requirements that permit adjusting course to avoid mistakes.
120819 -        ..
120820 -       The proposal claims SDS organic subject structures and the
120821 -       entire program environment permit flexible retrieval of
120822 -       information, reflective of human "intelligence."
120824 -        ..
120825 -       The SDS experiential record has been ported to the Internet as a
120826 -       means to deliver business intelligence.  The U.S. Army Corps of
120827 -       Engineers applied Com Metrics and reports the methodology saves
120828 -       money. The current proposal is to port the SDS search capability
120829 -       to the Internet for enhancing access to "intelligence" on the
120830 -       Internet, as defined in the proposal.
120832 -  ..
120833 - NSF guidelines suggest but do not require alignment of review with
120834 - proposals. ref DRT 1 1974  Limited time, and lack of experience using
120835 - alignment may have hampered NSF review, which cascaded into further
120836 - error.
120837 -
120838 -     1.  Panel's failure to cite the organizing methodology for
120839 -         information, and guidelines for applying "intelligence" on the
120840 -         Internet prevented understanding value added to exsiting
120841 -         methods.  For example, on 990715 Sally reported not having
120842 -         time to review the record online, ref SDS 37 0001, which
120843 -         reflects need for Judicious Review processes developed on
120844 -         990419, ref SDS 22 2320, and which is part of the proposal
120845 -         entirely overlooked in the NSF review. ref OF 15 0877
120847 -          ..
120848 -     2.  The conclusion that the proposal lacks a "systematic argument"
120849 -         for the need to add "intelligence" to management by providing
120850 -         automated integration of time and information is incorrect.
120851 -
120852 -         a.  First, the life experience of anyone involved with
120853 -             knowledge work, such as a reviewer, manager, scientist,
120854 -             doctor, mechanic, or engineer, attest to the need for
120855 -             "intelligence," as defined in the proposal. ref OF 14 0001
120857 -              ..
120858 -             This view was restrained by requiring access to POIMS on
120859 -             the Internet linked in the proposal for a full definition
120860 -             of "intelligence."  However, this was a deliberate design
120861 -             in the proposal aimed at providing experience with new
120862 -             ideas in order to facilitate understanding, as recommended
120863 -             by NSF staff on 990615. ref SDS 25 1044
120865 -              ..
120866 -             Part 2, Significance of the Innovation says... The nation
120867 -             will benefit from a direct, practical means to solve
120868 -             information overload, that otherwise overwhelms human span
120869 -             of attention, causing daily bumbling, mistakes, loss,
120870 -             crisis, conflict and disaster in public and commercial
120871 -             life, described by Henry Kissinger as an "Alice in
120872 -             Wonderland" environment. The explosion of communication in
120873 -             all forms adds urgency to this project. ref OF 14 0001
120875 -              ..
120876 -             The need to solve information overload is overwhelming. It
120877 -             is not an open question.
120879 -              ..
120880 -             The national initiative to cut medical mistakes reported
120881 -             on 991207, ref SDS 52 0001, and NASA's continual problems
120882 -             e.g., reported on 991001, ref SDS 49 0001, give visibility
120883 -             to a crushing problem every manager and every knowledge
120884 -             worker experiences every day.  These examples are not
120885 -             cited in the proposal, but evidence cultural awareness of
120886 -             pandemic bumbling caused by lack of "intellignece," as
120887 -             defined in the proposal.  Only the willfully blind are
120888 -             ignorant of this need.
120890 -              ..
120891 -         b.  The proposal cites the report by the U.S. Army Corps of
120892 -             Engineers on Oct 7, 1997 that "subjects" overlooked by
120893 -             managers due to limited span of attention cause rework
120894 -             that increase cost and delay, which is solved by
120895 -             Communication Metrics, using SDS organic subject
120896 -             structures. ref OF 14 0556
120898 -              ..
120899 -             Need to improve conventional subject and keyword methods
120900 -             are cited in the report. ref OF 14 1650
120902 -              ..
120903 -             ROI of 10:1 is cited as justification for porting the SDS
120904 -             organic subject structure to the Internet. ref OF 14 2124
120905 -
120906 -
120907 -
1210 -

SUBJECTS
Decision Support Systems
Bayseian Belief Networks
KM Dilemma
Knowledge Management Problem

2106 -
210701 -  ..
210702 - Knowledge Management Delimma Undefined by NSF
210703 - NSF Found Research Plan Defective, Principal Investigator Lacks Skill
210704 -
210705 - Panel concurs with the proposal that experts need to be interviewed,
210706 - but maintains this should be submitted in the proposal rather than
210707 - developed in research...
210708 -
210709 -     The researcher proposes to interview computer scientists as well
210710 -     as others in knowledge classification and formulate this into a
210711 -     phase two plan. While this is work that needs to be done, it would
210712 -     make a stronger proposal to have completed this prior to
210713 -     submitting the SBIR proposal. ref DRT 1 1118
210715 -  ..
210716 - Lead Reviewer comments...
210717 -
210718 -     ...the proposal presents no real plan either for analyzing the
210719 -     knowledge management dilemma... or for developing technologies
210720 -     and methods to respond,... it shows little awareness of research
210721 -     and practice in the several fields the proposal touches,
210722 -     including computational linguistics, search and retrieval
210723 -     systems, Bayesian belief networks, clustering algorithms, other
210724 -     neural network technologies, etc. ref DRT 1 6120
210725 -
210726 -         [...see below for more "computational linguistics."
210727 -         ref SDS 0 XN3M
210729 -          ..
210730 -         [On 000110 letter from Cornell University says Computational
210731 -         Linguistics does not support Com Metrics. ref SDS 56 0001]
210733 -          ..
210734 -         [On 000106 reviewed knowledge management dilemma with Morris
210735 -         at Intel. ref SDS 55 5120
210737 -          ..
210738 -         [On 000510 Doug Engelbart was denied NSF support on grounds
210739 -         that "knowledge" was not defined in proposal. ref SDS 62 0004
210741 -          ..
210742 -         [On 011222 "paraphrasing" presents overwhelming KM dilemma
210743 -         that SDS solves. ref SDS 68 XT4K
210745 -          ..
210746 -         [On 060211 Open Iris new software release by SRI that provides
210747 -         clustering support that may implement solutions proposed today
210748 -         by NSF for intelligence support of Communication Metrics.
210749 -         ref SDS 70 QW6O
210751 -      ..
210752 -     The principal investigator... does not show evidence of having the
210753 -     intellectual and experiential tools required to approach those
210754 -     issues with anything like sufficient polish or rigor.  His reach
210755 -     exceeds his grasp. ref DRT 1 5822
210756 -
210757 -         [On 991221 Sally Nerlove at NSF reported that Welch is a
210758 -         Diamond in the Rough. ref SDS 54 0001]
210760 -  ..
210761 - Analysis...
210762 -
210763 -     1.  Asking for research to be included in the proposal puts the
210764 -         "cart before the horse," as stated in Part 2 of the proposal.
210765 -         ref OF 14 3740  This was discussed with Sally on 990629.
210766 -         ref SDS 28 0886
210768 -          ..
210769 -     2.  The proposal states that SDS solves the knowledge management
210770 -         dilemma by automated integration of time and information,
210771 -         ref OF 14 7802, which improves alphabet technology, and notes
210772 -         that part of SDS has been ported to the Internet where the
210773 -         implementation can be observed, which thereon establishes
210774 -         credentials to perform.
210776 -            ..
210777 -           [On 011222 "paraphrasing" presents overwhelming KM dilemma
210778 -           that SDS solves. ref SDS 68 XT4K
210780 -          ..
210781 -         SDS on the Internet demonstrates success accomplishing a level
210782 -         of tool support for "intelligence" that nobody else has
210783 -         accomplished using methods cited in the review. ref DRT 1 6120
210784 -         NSF reviewers cite no applications that support Enterprise
210785 -         Management with skills and methods claimed to be needed for
210786 -         solving the knowledge management delimma, as grounds for
210787 -         criticizing the Welch proposal. ref SDS 0 0844
210789 -          ..
210790 -         NSF erred in failing to click on a link to discover that
210791 -         Communication Metrics is not simply a theory, but that most of
210792 -         it already exists, and thereon demonstrates ability to perform
210793 -         beyond the capability of other grantees who have been funded,
210794 -         but have not produced useful knowledge capability.
210796 -          ..
210797 -         NSF further erred applying artificial intelligence techniques
210798 -         to evaluate Com Metrics.
210800 -          ..
210801 -         Bayseian belief networks are graphical representations of
210802 -         probabilistic relationships for reasoning under uncertainty
210803 -         using mathematical modeling that apply expert opinion about
210804 -         the dependencies between variables. ref OF 6 2405  BBNs have
210805 -         been used for medical diagnosis and analysis of mechanical
210806 -         failures, ref OF 6 7144, for safety litigation, military
210807 -         vehicle and software reliablity. ref OF 6 1760  Help systems
210808 -         in Microsoft Office; also the ‘intelligent’ printer
210809 -         fault diagnostic system which you can run when you log onto
210810 -         Microsoft’s web site use BBN to resolve printer problems.
210811 -         ref OF 6 7144
210813 -          ..
210814 -         These applications do not align with the Welch proposal to use
210815 -         technology for aiding human thinking by enhancing the alphabet
210816 -         for adding "intelligence" to management.
210817 -
210818 -             [On 000106 discussed SDS for computer aided thinking.
210819 -             ref SDS 55 1316]
210821 -              ..
210822 -             [On 010217 received question from Benham Grey with KM
210823 -             section of Brint, asking about correlation between POIMS
210824 -             technology explained on Welchco.com, and AI capability
210825 -             using Bayesian statistics and neural networks to provide a
210826 -             super search engine based extracting patterns. ref SDS 66
210827 -             5T8K
210829 -          ..
210830 -         The U.S. Air Force used statistical modeling based on expert
210831 -         opinion in hopes of preventing management information from
210832 -         degrading toward entropy over time, reported on 970707.
210833 -         ref SDS 12 0108  This method failed.  On 971008 the U.S. Army
210834 -         Corps of Engineers reported Communication Metrics succeeded.
210835 -         ref SDS 13 2710  By what reasoning can a method that has not
210836 -         worked be the basis to criticize a method that has been found
210837 -         to save money?
210839 -          ..
210840 -     3.  The panel's determination that the research plan is weak
210841 -         aligns with Mark Haselkorn's comment on 990729. ref SDS 43
210842 -         0001
210844 -          ..
210845 -         Weak research planning is not dispositive of whether the
210846 -         project can succeed based on two facts...
210847 -
210848 -         a.  Knowledge management is frontier, cutting edge science.
210849 -             Nobody knows what it means.  Most breakthroughs do not
210850 -             occur in the lab, but in the field of life where chance
210851 -             arrangements augment capability and subsequent experience
210852 -             leads to scientific theories that explain augmentation, as
210853 -             set out by Dr. Thomas K. Landauer in his book...
210854 -
210856 -                        ..
210857 -                       The Trouble with Computers
210858 -
210859 -
210860 -             ...reviewed on 950710. ref SDS 4 8303
210862 -              ..
210863 -             Intial research for knowledge management entails rough
210864 -             approximations of conventional practice in order to form
210865 -             the foundation for rigorous work.
210867 -              ..
210868 -         b.  SDS already exists, proving the theory of POIMS technology
210869 -             for augmenting human intelligence. ref OF 1 0561 However,
210870 -             weak the research plan, this foundation is far ahead of
210871 -             anything NSF might fund for this purpose.  Funding is to
210872 -             expand and sharpen subject identification, and make a
210873 -             stronger implementation on the Internet, so it is easier
210874 -             for more people to use, and therefore more commercially
210875 -             viable.  That is the objective of the NSF program for
210876 -             SBIR, i.e., to make innovation profitable.
210878 -              ..
210879 -             We want to bridge the gap between technical feasiblity, of
210880 -             POIMS technology, established by SDS, and commercial
210881 -             viability.
210882 -
210883 -
210884 -
210885 -
210886 -
210887 -
210888 -
2109 -

SUBJECTS
Decision Support Systems
Computational Linguistics
Speech Recognition
Bayesian Decision Support
Bayesian Statistics Neural Networks Search Engine Extracts Patterns

3007 -
300801 -          ..
300802 -     4.  Computational Linguistics combines humanities, natural and
300803 -         behavioural sciences, and engineering to support artificial
300804 -         intelligence (AI), cognitive science, and speech specialists
300805 -         with design and analysis of natural language processing
300806 -         systems.  Language pattern matching applications include
300807 -         grammar checkers for wordprocessing, speech and handwritting
300808 -         recognition, and language translation. ref OF 5 3994
300810 -          ..
300811 -         NSF ideas on using computational linguistics may be related to
300812 -         lead review discussion of Bayesian belief networks, per above.
300813 -         ref SDS 0 0844
300815 -          ..
300816 -         On 980226 IBM announced progress developing "data mining"
300817 -         using language pattern matching, which may be based on
300818 -         computational linguistics cited by NSF today. ref SDS 14 1247
300820 -            ..
300821 -           [On 000110 letter from Cornell University says Computational
300822 -           Linguistics does not support Com Metrics. ref SDS 56 0001]
300824 -            ..
300825 -           [On 000314 cited language pattern matching to support
300826 -           alignment, as possible research with LANL. ref SDS 58 2838
300828 -            ..
300829 -           [On 000424 Sandy Klausen suggests this is purpose of DKR
300830 -           project. ref SDS 59 4212
300832 -            ..
300833 -           [On 000428 Henry Liu may be able to support this task.
300834 -           ref SDS 60 5358
300836 -            ..
300837 -           [On 000503 Jack Park wants to pursue this task. ref SDS 61
300838 -           6860
300840 -            ..
300841 -           [On 000623 Jack proposes engine to map subjects. ref SDS 63
300842 -           2915
300844 -            ..
300845 -           [On 001025 Paul Fernhout explains challenge of identifying
300846 -           knowledge representation well enough to accomplish useful
300847 -           automated organization of information. ref SDS 65 LR6N
300849 -            ..
300850 -           [On 010217 received question from Benham Grey asking about
300851 -           correlation between POIMS technology explained on
300852 -           Welchco.com, and AI capability using Bayesian statistics and
300853 -           neural networks to provide a super search engine based
300854 -           extracting patterns. ref SDS 66 5T8K
300856 -            ..
300857 -           [On 060211 Open Iris new software release by SRI that
300858 -           provides clustering support that may implement solutions
300859 -           proposed today by NSF for intelligence support of
300860 -           Communication Metrics. ref SDS 70 QW6O
300862 -          ..
300863 -         Pyscholinguistics uses mathamatical modeling to examine
300864 -         cognitive processes of language. ref OF 5 5609
300866 -          ..
300867 -         On 960518 reviewed Doctor Thomas K. Landauer's mathamatical
300868 -         model for Latent Semantic Analysis (LSA) uses computational
300869 -         linguistics to analyse learning processes that show word
300870 -         meanings constantly change in the mind. ref SDS 7 3734
300871 -         Computational linguistics offers no solution.  The Welch
300872 -         proposal explains a method of Communication Metrics for
300873 -         maintaining alignment to prevent meaning drift. ref OF 14 5561
300875 -          ..
300876 -         POIMS and the New World Order... papers define "intelligence"
300877 -         and explain how technology can do something useful to help
300878 -         people and organizations perform knowledge work, cited in the
300879 -         proposal. ref OF 14 1148
300881 -          ..
300882 -         This record supports continued effort.
300884 -  ..
300885 - NSF Practice Justifies Support for Communication Metrics
300886 -
300887 -         Review shows NSF has made awards for information research,
300888 -         which have not advanced knowledge management, because they do
300889 -         not define "knowledge" and "intelligence" in a way that can be
300890 -         aided by technology, which has been accomplished by POIMS.
300891 -         ref OF 1 0561
300893 -          ..
300894 -         None of the abstracts show awareness of how to improve
300895 -         communication, management and earnings; none mention the need
300896 -         to solve limited span of attention, nor to support alignment
300897 -         to avoid meaning drift, explained in POIMS. ref OF 1 8774
300899 -          ..
300900 -         NSF's web resources
300901 -
300902 -
300903 -                    http://www.nsf.gov/verity/srchawd.htm
300905 -  ..
300906 -         ...show field search...
300907 -
300908 -
300909 -                    http://www.nsf.gov/verity/srchawdf.htm
300911 -  ..
300912 -         ...and the list at....
300913 -
300914 -
300915 -                   http://www.nsf.gov/cgi-bin/verity/srchawdf
300916 -
300917 -
300918 -           (access is very, very slow.)
300920 -          ..
300921 -         NSF approved award 9216848 on July 1, 1998 for...
300922 -
300923 -              A Scientific Group Communications & Collaborative Testbed
300924 -              for Upper Atmospheric Research...
300925 -
300926 -
300927 -                  http://www.nsf.gov/cgi-bin/showaward?award=9216848
300929 -          ..
300930 -         The abstract says this award is for...
300931 -
300932 -              ... a multidisciplinary effort linking research in
300933 -              computer science, behavioral science, and upper
300934 -              atmospheric and space science to build a prototype system
300935 -              for a distributed but shared working environment: the
300936 -              vision of a collaboratory.
300938 -               ..
300939 -              This research will [study] collaboration tools... The
300940 -              Sondre Stromfjord researchers expect to obtain greater
300941 -              efficiency in joint experimental operations, analysis,
300942 -              and discovery by guiding the requirements for the
300943 -              collaboration environment which they will utilize to
300944 -              support their research.
300946 -          ..
300947 -         This proposal is silent on the need to integrate time and
300948 -         information, which is the only way to grow new knowledge.
300949 -         Further, collaboration requires building and maintaining
300950 -         shared meaning. Project 9216848 is silent on solving meaning
300951 -         drift.
300953 -          ..
300954 -         NSF approved award 9812755 for a project that expires on
300955 -         August 31, 2000 for...
300956 -              ..
300957 -              Enhancing Robustness of Information through
300958 -              Distributed Adaptive Coordination
300959 -
300961 -         ..
300962 -        http://www.nsf.gov/search97cgi/vtopic?action=View&VdkVgwKey=http%3A%2F%2Fwww%2Einteract%2Ensf%2Egov%2Fcise%2Fabst%2Ensf%2FPages%2FC521F7713038AE55852567D700693B74&DocOffset=13&DocsFound=182110&SearchUrl=http%3A%2F%2Fwww%2Ensf%2Egov%2Fsearch97cgi%2Fvtopic%3FQueryText%3D%26ResultStart%3D11%26ResultCount%3D10&
300964 -          ..
300965 -         Abstract says this award is for...
300966 -
300967 -              ...[creating] software for complex and large-scale
300968 -              information systems... [Development entails] potentially
300969 -              hundreds of intelligent real-time agents.  Robust
300970 -              information systems will adapt... in response to changes
300971 -              in system goals, ... requirements... A multi-agent <8232
300972 -              architecture appropriate for such systems is being
300973 -              developed ... for... scheduling, ... coordination, ...
300974 -              design, ... inappropriate behavior... and on-line <1575
300975 -              learning to improve the performance... The impact of this
300976 -              work is to provide software, techniques and empirical
300977 -              studies that will facilitate the construction of the next
300978 -              generation of robust information systems.
300979 -              http://dis.cs.umass.edu/
300981 -          ..
300982 -         This program sounds similar to Stanford's work on mediators,
300983 -         using "intelligent" agents, reviewed on 940603, ref SDS 2 9861
300984 -         which is cited in POIMS.
300986 -          ..
300987 -         NSF approved award 9703470 for a project that expires on
300988 -         July 31, 2000 for...
300989 -
300990 -              CISE Research Infrastructure: A Next Generation Computing
300991 -              and Communications Substrate
300993 -         ..
300994 -        http://www.nsf.gov/search97cgi/vtopic?action=View&VdkVgwKey=http%2F%2Fwww%2Einteract%2Ensf%2Egov%2Fcise%2Fabst%2Ensf%2FPages%2FC5035D14D95C2094852567D7006868DB&DocOffset=18&DocsFound=182110&SearchUrl=http%3A%2F%2Fwww%2Ensf%2Egov%2Fsearch97cgi%2Fvtopic%3FQueryText%3D%26ResultStart%3D11%26ResultStart%3D11%26ResultCount%3D10&
300996 -          ..
300997 -         Abstract says this award is for...
300998 -
300999 -              ...[developing] a new computing and communications
301000 -              substrate that provides "access to anything, anytime,
301001 -              anywhere". To achieve this goal Cornell researchers plan
301002 -              to: (1) do fundamental new research, (2) develop
301003 -              integrated systems to demonstrate the power of their
301004 -              results, and (3) test these systems. In order to develop
301005 -              the proposed systems, it is critical to test them with
301006 -              real systems and real users.  These researchers plan to
301007 -              use their teaching activities as a ready source of users
301008 -              to test their systems, as well as using the systems to
301009 -              disseminate their research results and to provide
301010 -              experimental information services. Cornell University is
301011 -              moving towards a campus-wide hybrid ATM and switched
301012 -              ethernet communications network, which will eventually
301013 -              enable them to test their systems with campus-wide
301014 -              network of 20,000 users. They also expect to release
301015 -              their systems for use by other academic institutions and
301016 -              businesses.
301018 -          ..
301019 -         Anytime, anywhere access to "intelligence" is accomplished by
301020 -         Communication Metrics...
301021 -
301022 -
301023 -                           http://www.welchco.com/
301025 -          ..
301026 -         The need for wide-scale testing is correct.  Educational
301027 -         testing was proposed for Communication Metrics to the
301028 -         University of Santa Clara on 951012. ref SDS 5 1940  On 970328
301029 -         U.S. Army Corps of Engineers reported Communication Metrics
301030 -         delivered useful "intelligence" that improved management.  On
301031 -         971007 the Corps reported cost benefit of Com Metrics was
301032 -         10:1, supporting prior limited testing by PG&E.  These
301033 -         successful limited tests support wider testing of Com Metrics,
301034 -         based on NSF award 9703470.
301036 -          ..
301037 -         NSF approved award 9616254 for a project that expired on
301038 -         October 31, 1999 for...
301039 -
301040 -              Efficient Algorithms for Learning and Reasoning from
301041 -              Data
301043 -          ..
301044 -         The abstract says this award is for...
301045 -
301046 -              ... development of ... algorithms that allow learning and
301047 -              limited reasoning over very large repositories of data.
301048 -              The goal is to combine these algorithms into an
301049 -              environment that supports summarization of very large
301050 -              streams of data (data mining) and the drawing of
301051 -              reasonably accurate conclusions from these summaries...
301052 -              (3) Incorporating knowledge in memory-based reasoning
301053 -              methods. The availability of high-speed, reliable,
301054 -              intelligent algorithms for learning and reasoning from
301055 -              data will have important applications in digital
301056 -              libraries, medical care, process control, and resource
301057 -              allocation, among other areas.
301058 -
301059 -
301061 -  ..
301062 - Appeal Procedures for NSF Grant Determinations
301063 -
301064 - Section 4.3 says...
301065 -
301066 -    Phase I proposals that have been declined or returned by NSF are
301067 -    not eligible for reconsideration under the same Program
301068 -    Solicitation, however, they can be resubmitted after suitable
301069 -    revision, under subsequent Solicitations. Phase II proposals that
301070 -    have been declined are not eligible for resubmission. ref OF 9 3068
301072 -  ..
301073 - This seems to indicate there is no procedure for NSF to correct
301074 - mistakes in the review process.
301075 -
301076 -     [On 991221 Sally suggests submitting the proposal next year.
301077 -     ref SDS 54 0001]
301078 -
301079 -
301080 -
301081 -
301082 -
301083 -
301084 -
301085 -
301086 -
301087 -
301088 -
301089 -
3011 -
Distribution. . . . See "CONTACTS"