THE WELCH COMPANY
440 Davis Court #1602
San Francisco, CA 94111-2496
415 781 5700


S U M M A R Y


DIARY: February 5, 2001 05:19 PM Monday; Rod Welch

Fix problem with SDS Full Subject report on new computer c13.

1...Summary/Objective
.....Turn Diskpref Off:
.....DMA/UDMA and Win2000 - Changed CD ROM from PIO to DMA
.....Turn Off CD Auto Play Win2000
2...06112 Call Macro 120 Evaluate Report Period
3...25.bat Global Pointers
4...000007 Macro 120
5...061122 Configure Sort Command File


..............
Click here to comment!

CONTACTS 

SUBJECTS
Subject, 06112
Macro 120 Process Report Spec
061122 Selection Command File, Configure
Performance Slower for SDS Reports than C11
Supersort, 910306

0807 -    ..
0808 - Summary/Objective
0809 -
080901 - Follow up ref SDS 29 0000, ref SDS 28 0000.
080902 -
080903 - C12 takes 1 second, and c13 takes 10 seconds to process the same
080904 - information.  Modified the code that builds the pointer file so it is
080905 - more efficient to select pointers.  This did not change the
080906 - performance disparity. ref SDS 0 EZ4H  Got some suggestions from
080907 - Morris indicating disk caching is an issue, but there does not seem to
080908 - be a procedure for configuring buffers or disk caching on c13 running
080909 - w2k, that are available in w98. ref SDS 0 YU9K  Made changes to w2k
080910 - for tracking disk performance, ref SDS 0 1J7L, and setting IDE for the
080911 - CD ROM drive to DMA. ref SDS 0 AJ4K  Also, turned off auto-play for
080912 - the CD ROM drive. ref SDS 0 LJ8I
080913 -
080914 -     [On 010206 w2k crashed on c13. ref SDS 32 0001
080915 -
080916 -     [On 010206 got information from MS. ref SDS 33 0001
080917 -
080918 -
080919 -  ..
0810 -
0811 -
0812 - Problem
0813 -
081301 - A subject for a long period, i.e., greater than 4 weeks, seems to be
081302 - broke running on c13, reported on 010202. ref SDS 30 QL5K  Found out
081303 - the problem is the sort and selection operation.
081304 -
081305 -     [On 010206 new, more severe system problem encountered.
081306 -     ref SDS 32 ML5J
081307 -
081308 - For example, c12 is a notebook computer with a 650 MHz processor and
081309 - a low internal disk.  It executes a long period report in 30% of the
081310 - time required for c13 which has a 1.4 GHz processor, a 400 MHz bus,
081311 - and a high speed 7200 RPM disk.
081312 - ..
081313 - Testing Plan......
081314 -
081315 -        Run a search at the same time, key for key, on c13 and c12 to
081316 -        identify where processing changes on c13.
081317 -
081318 -        Set up an identical report spec.
081319 -
081320 -        F7 sets process in debug mode.
081321 -
081322 -
081323 - Background.
081324 -
081325 -     On 930430 SDS performance was addressed as a disk caching problem.
081326 -     ref SDS 5 JY9O
081327 -     ..
081328 -     On 920115 example of using smartdrv.sys. ref SDS 2 0001
081329 -
081330 -     On 920722 loading mouse driver high. ref SDS 2 L3W2
081331 -
081332 -         Looks useful for smartdrv also. ref SDS 2 FL9L
081333 -
081334 -         need to load emm386 in order to load stuff high.
081335 -
081336 -     On 920725 using smartdrv. ref SDS 3 CI5M
081337 -
081338 -     On 940607 solving SDS reports failing. ref SDS 7 0003
081339 -
081340 -     On 950507 history of slow reporting was reviewed, diskcaching
081341 -     with smartdrv reported solved problem. ref SDS 8 0001
081342 -     ..
081343 -     On 960111 mode80 was used for something. ref SDS 9 0001
081344 -
081345 -     On 970729 very similar issue. ref SDS 19 CP4L
081346 -
081347 -     On 970805 similar issue. ref SDS 21 0001
081348 -
081349 -
081350 -
0814 -
0815 -
0816 - 2121 called Morris
0817 -
081701 - This follows up the telecon last night, when he suggested compressing
081702 - the drives.  Today, this was done after purchasing and installing a
081703 - version of Diskeeper, ref SDS 31 OJ3O, and finding that this did not
081704 - improve performance of c13.
081705 - ..
081706 - He suggested there could be a virus checker operating in the
081707 - background that is consuming processor time on c13.  We could not find
081708 - evidence of that.
081709 -
081710 - Morris suggested opening....
081711 -
081712 -    Control Panel
081713 -      System Properties
081714 -         Performance
081715 - ..
081716 - Morris said w98 on his computer has an option for File System
081717 - where he set a Read Ahead parameter that improves disk caching.
081718 -
081719 - This feature is not available for w2k.  Performance only has two
081720 - options...
081721 -
081722 -           Applications
081723 -
081724 -           Background Services
081725 -
081726 - Another option is...
081727 -
081728 -           Virtual memory
081729 -
081730 -           Total paging file size
081731 - ..
081732 - Morris feels there is a settings issue based on c13 taking twice
081733 - as long for an operation as c12, which has a processor that is 70%
081734 - slower.
081735 -
081736 - We looked on the Internet for information on how to optimize w2k for
081737 - the Pentium 4 processor and the D850GB mother board system.
081738 -
081739 - Perhaps the 400 MHz bus is an issue.
081740 -
081741 -  ..
081742 - We found a source that suggested....
081743 -
081744 -        http://win2000tips.home.att.net/Tipstricks.htm#Turn Diskpref Off:
081745 -
081746 -      ..
081747 -     Turn Diskpref Off:
081748 -
081749 -     By default, Win2k is set to collect physical drive data.
081750 -
081751 -     As home users we don't need this.
081752 -
081753 -     To disable the disk performance counters:
081754 -
081755 -     Start
081756 -     Programs
081757 -     Command Prompt
081758 -     diskperf -n
081759 -     reboot
081760 -     ..
081761 -     If you ever want to return to default:
081762 -
081763 -                 diskperf -yd
081764 -
081765 -        [On 010206 w2k failed; changed setting back. ref SDS 32 ML5J
081766 -
081767 -
081768 - Same source suggested....
081769 -
081770 -
081771 -        http://win2000tips.home.att.net/Tipstricks.htm#DMA/UDMA and Win2000
081772 -
081773 -
081774 -      ..
081775 -     DMA/UDMA and Win2000 - Changed CD ROM from PIO to DMA
081776 -
081777 -     Odds are that if you were using the DMA setting successfully for
081778 -     your HD's in Win98 it set up properly in Win2000.  However, the CD
081779 -     drive is now, more then likely, in the PIO mode.
081780 -
081781 -     To check if your drives are in the DMA mode do this:
081782 -
081783 -     Right click on My Computer
081784 -     Select Manage
081785 -     Device Manager
081786 -     IDE ATA/ATAPI Controller
081787 -     Primary IDE Channel
081788 -     Advance Setting Tab
081789 -
081790 -        [On 010206 w2k failed; changed setting back. ref SDS 32 ML5J
081791 -     ..
081792 -     It will show the current transfer mode.  If it is not
081793 -     correct try Auto Detect.  Reboot and check it again.  If it still
081794 -     isn't correct for your drives try selecting DMA mode and then
081795 -     rebooting. Take a look also at the Secondary IDE Channel.
081796 -     Normally this would be where you have your CD drives connected.
081797 -     If they show PIO try changing them to DMA. (The DMA mode will not
081798 -     make the CD drives run any faster but will greatly reduce the CPU
081799 -     load from around 90% to about 10% or lower which will allow other
081800 -     programs running at the same time to run faster.)
081801 -
081802 - In an effort to improve system performance, made change from PIO to
081803 - DMA for secondary IDE Channel.
081804 -
081805 -
081806 -
081807 -
0819 -

SUBJECTS
CD ROM Autorun Disable/Enable
Autorun Disable/Enable

1005 -
100501 -  ..
100502 -
100503 - Made this change for convenience....
100504 -
100505 -
100506 -       http://win2000tips.home.att.net/Tipstricks.htm#Turn Off CD Auto Play Win2000
100507 -
100508 -
100509 -      ..
100510 -     Turn Off CD Auto Play Win2000
100511 -
100512 -     Follow up ref SDS 27 9GK7.
100513 -
100514 -     This may supplement instructions for changing the registry on
100515 -     990825. ref SDS 27 9GK7
100516 -
100517 -          [On 020416 got additional instructions for modifying the
100518 -          registry. ref SDS 37 TM6K
100519 -
100520 -     To disable AutoPlay
100521 -
100522 -     1.  Double-click System Management (My Computer)
100523 -         ..
100524 -     2.  On the Tools menu, click Folder Options, and then click
100525 -         the File Types tab.
100526 -         ..
100527 -     3.  In Registered file types, click AudioCD, and then click
100528 -         Advanced.
100529 -
100530 -     4.  In Actions, click Play, and then click Set Default.
100531 -
100532 -     Note:
100533 -
100534 -     a.  You can temporarily disable AutoPlay by pressing SHIFT while
100535 -         you insert an audio CD.
100536 -
100537 -     b.  Repeating steps 1 through 4 enables AutoPlay.
100538 -
100539 -     c.  When Play is displayed in bold, AutoPlay is enabled. When Play
100540 -         is not displayed in bold, AutoPlay is disabled.
100541 - ..
100542 - Actually, was not able to get this to work, but holding the
100543 - shift key down while a CD is installed does seem to work for CDs that
100544 - have music.  Other CDs for example to install programs, do not seem to
100545 - respond to holding the shift key.
100546 -
100547 - May still be able to accomplish the same thing using instructions on
100548 - 990825. ref SDS 27 9GK7
100549 -
100550 -
100551 -
100552 -
100553 -
100554 -
1006 -
1007 -
1008 - 835P Progress
1009 -
100901 -  ..
100902 - 06112   Call Macro 120 Evaluate Report Period
100903 -
100904 - Follow up ref SDS 29 7966, ref SDS 26 7966.
100905 -
100906 - Line 240, ref OF 1 6612, -label gbLs is where macro 120 is called,
100907 - which is executed in 000007.
100908 -
100909 - Line 240, ref OF 1 6612, -label gbLs, this is where macro 120
100910 - executes.  Everything is the same to this point.
100911 - ..
100912 - Line 50, ref OF 1 TQ6K, the long term selection spec with the
100913 - command file seems to be the hang up....
100914 -
100915 -   s i 360 cr;no-r;s e:\sd\08\00000\06\00001;o 1Y001 k-out;no-o;cf 1z
100916 -
100917 - ...where 1z is.....
100918 -
100919 -    se ( fi 1 6 bt "990106","991231" or fi 1 6 bt "000101","010205") &
100920 -    and fi 75 88 ="03 00050 01 02" &
100921 -     and fi 15 19 = "00101"
100922 -    k=1 19 220 225 23 24;g
100923 -
100924 - This spec on c12 takes 1 second to process following....
100925 -
100926 -     Warning SW5: Insufficient fields or columns for key or select
100927 -                  in one or more records, blanks assumed
100928 -
100929 -     99919 records input for sort
100930 -         99469 sort records excluded or not selected
100931 -     450 records sorted
100932 -     Output file size 13050 bytes
100933 -
100934 -     ..
100935 -     Sort/merge completed in 10 seconds.
100936 -
100937 -   [On 010212 after a week of different stuff, problem not improved in
100938 -   the least. ref SDS 35 0001
100939 -
100940 - Same spec on c13 takes 10 seconds, 10 times longer....
100941 -
100942 -     Warning SW5: Insufficient fields or columns for key or select
100943 -                  in one or more records, blanks assumed
100944 -
100945 -     98840 records input for sort
100946 -         98405 sort records excluded or not selected
100947 -     435 records sorted
100948 -     Output file size 12615 bytes
100949 -
100950 -     ..
100951 -     Sort/merge completed in 10 seconds.
100952 -
100953 -
100954 -
100955 -
1010 -

SUBJECTS
25.bat Global Pointers Archive SDS Records
Summary pointers, 910304
25.bat Archive Pointers Using OTS for Sort, Select

1306 -
130601 -  ..
130602 - 25.bat  Global Pointers
130603 -
130604 - To fix the problem of slow performance on c13, reported on 010202,
130605 - ref SDS 30 QL5K, we may be able to help the 1.4 GHz processor catch up
130606 - with the 650 MHz processor, by creating a command file that sorts the
130607 - pointers and removes extraneous entries, that occassionally get into
130608 - the subject pointer file.
130609 -
130610 -
130611 -                          25.bat
130612 -
130613 -
130614 - ...is the operation that creates pointers.  It was last modified on
130615 - 980423. ref SDS 22 0001
130616 - ..
130617 - On 970729 similar problem was caused by lack of buffers=99 in
130618 - config.sys. ref SDS 19 CP4L
130619 -
130620 -     [On 010207 learned w2k has config.nt file; tried buffers=99, but
130621 -     did not solve problem.
130622 -
130623 -     [On 010302 OTS replaced Supersort.
130624 -
130625 - On 970804 same problem on this issue. ref SDS 20 0001
130626 -
130627 - On 970805 tried to solve problem by setting disk cache params.
130628 - ref SDS 21 0001
130629 -
130630 - Can run a sort after all pointers are copied together....
130631 - ..
130632 - Currently the code...
130633 -
130634 -     copy \kq\* /a e:\sd\08\00000\06\00001 >nul
130635 -
130636 - ...which just appends all the subject pointers into a big file.
130637 -
130638 -
130639 -  ..
130640 - We can instead put them in a big temporary file.
130641 -
130642 -
130643 -     copy \kq\* /a docs0 >nul
130644 -
130645 -
130646 - ...then sort that file and put the contents in the subject pointer
130647 - file.
130648 -
130649 - Will create command file....
130650 -
130651 -
130652 -                      c: sd 03 02701
130653 -
130654 - ..
130655 - Need to know how to specify a select command.  The record on
130656 - 910321 gives an example. ref SDS 1 M64J
130657 -
130658 - Think we can streamline by sorting the subject pointer file.
130659 -
130660 -
130661 -     c:\sd\01\s i 120 cr;s docs0; cf c:\sd\03\02701
130662 -
130663 -
130664 - ....where 02701 has....
130665 -
130666 -        se fi 28 31 = "K101"
130667 -        o e:\sd\08\00000\06\00001
130668 -
130669 -          Sort subjects by user by date
130670 -
130671 -        k 75 115 15 19 1 13
130672 -        go
130673 - ..
130674 - After making these changes, was able to create a compact pointer
130675 - file that contains only valid selection data, and sorted in a way that
130676 - all subject code strings are assembled together for each user, which
130677 - should speed up the report opertion.
130678 - ..
130679 - However, it didn't solve the problem.
130680 -
130681 - C12 which has 650 MHz still creates a report in about 4 seconds, and
130682 - c13 with a 1.4 GHz processor takes about 15 seconds.
130683 -
130684 -     This seems to be a Supersort problem with selection.  Similar
130685 -     problems occurred with Supersort on 930244, ref SDS 4 OV5L, and on
130686 -     970729, when buffers=99 in config.sys solved problem. ref SDS 19
130687 -     CP4L
130688 -
130689 -       [On 010210 got lead for help with Supersort. ref SDS 34 0001
130690 -
130691 -
130692 -  ..
130693 - 000007  Macro 120
130694 -
130695 - Line 1050, ref OF 4 SK5K, calls macro 120.
130696 -
130697 - Line 1310, ref OF 4 AM6I, is about where the code begins to execute
130698 - in the macro file.
130699 -
130700 -    After macro 120 everything looks the same.
130701 -
130702 -    The cursor for both c12 and c13 is at the end of the report spec
130703 -    generated by macro 120, and the format looks identical.
130704 -
130705 - Line 1480, ref OF 4 6290, -label cOOKay code here calls a separate
130706 - command processor to configure a search spec.  This was done to
130707 - reduce the size of the command processor.
130708 - ..
130709 - No changes were needed in this routine.
130710 -
130711 -
130712 -
130713 -  ..
130714 - 061122  Configure Sort Command File
130715 -
130716 - Line 40, ref OF 2 OZ6H, sets global 91 0, and sets global 91 888, if
130717 - 91 = 888
130718 -
130719 - Everything seems to process the same.
130720 -
130721 -
130722 -
130723 -
130724 -
130725 -
130726 -
130727 -
130728 -
130729 -
130730 -
130731 -