THE WELCH COMPANY
440 Davis Court #1602
San Francisco, CA 94111-2496
415 781 5700
rodwelch@pacbell.net


S U M M A R Y


DIARY: November 28, 2003 08:11 AM Friday; Rod Welch

SDS save F2 functions added support for formatting and memory.

1...Summary/Objective
2...References Processed in Fresh Session Successful but Districting
3...0702051 SDS Format Flexible Structure for Control Fields
4...Reference Data Bases Processed in Separate Memory Segments
5...0702051 F2 Save Windows Start Nested Application Reference Pointers


..............
Click here to comment!

CONTACTS 

SUBJECTS
Format Structure Provide Flexible Consistency Line Numbers   cord Seg
Format Line Numbers SDS Records Flexible Structure Knowledge Space Ha
Follow Up Format Fails for Summary/Objective When There is an Anchor
Format Reference Fields Verify Correct Structure During F2 Save Op wi
Control Field Header Line Numbers above Control Fields for Processing

0807 -
0807 -    ..
0808 - Summary/Objective
0809 -
080901 - Follow up ref SDS 6 0000, ref SDS 3 0000.
080902 -
080903 - More work on consistent application of flexible structure in Knowledge
080904 - Space.  Reduced keystrokes and investing time paying attention to
080905 - details of SDS record structure.  Encountered a strange occurrance of
080906 - out of memory using the F2 save function, so changed the design to add
080907 - another level of processing in a separate memory segment for updating
080908 - reference data bases. ref SDS 0 EG8G  This seems to solve the problem,
080909 - but introduces a new problem because now there is a noticable delay
080910 - due to loss of screen control using F2 to save the record. ref SDS 0
080911 - HH9N
080912 -
080913 -
080914 -
080916 -  ..
0810 -
0811 -
0812 - Evaluation
0813 -
081301 - References Processed in Fresh Session Successful but Districting
081302 -
081303 - Vigorous testing indicates changes to improve memory were effective.
081304 - ref SDS 0 P44V
081306 -  ..
081307 - Added support on formatting reduces keystrokes and mistakes that take
081308 - time to recover.  This comes at a cost of increased time to save
081309 - records.
081311 -  ..
081312 - Increased capacity using the Windows Start command for separate memory
081313 - segments to update SDS data bases for references, developed on 021125,
081314 - ref SDS 0 1K3L, also, takes more time and causes loss of screen
081315 - stability.  Added time is not noticable on small records, but on large
081316 - records, like the record on 010725, ref SDS 2 0001, there is pause of
081317 - about 2 seconds in the work using a 1.4 GHz system.  On slower
081318 - systems, like the c12 notbeook running at 600 MHz, the delay is about
081319 - 4 seconds on this example.
081321 -  ..
081322 - Adding four (4) more Start commands for processing in separate memory
081323 - segments causes noticable flicker in the screen, rather than maintain
081324 - a steady screen for user interface.  The problem is most sever on c12,
081325 - which is a notebook computer and so has a much slower data transfer
081326 - rate between memory and the disk.  Increasing the number of disk reads
081327 - and writes therefore causes severe changes in the size of the screen.
081328 - On c12 the screen visibly collapses and reforms multiple times during
081329 - the F2 save operation.
081331 -  ..
081332 - Would like to figure out a way to eliminate delay saving the record
081333 - and to maintain screen stability, rather than present a jumpy black
081334 - screen to the user with a long pause to save a record.
081335 -
081336 -        [On 031203 reported completion of work revising design of F2
081337 -        save process in 0702051 to increase speed and maintain screen
081338 -        stability, while retaining memory capacity developed today.
081339 -        ref SDS 9 0001
081341 -  ..
081342 - One solution is not to call macro 142 op routinely, but to provide an
081343 - alternate method that simply writes the record to the disk.  The F1 F2
081344 - function does this.  Could revise the scheme to call the code for the
081345 - F1 F2 method when F2 is hit twice within 1 second.  This would free
081346 - up F1 F2 for something else.  The first option could be to only save
081347 - the record, and the full save function could occur only when the User
081348 - expressly calls it.  Since 04702 runs 0702051 when the record is
081349 - archived, there would be at least one occassion when 0702051 would
081350 - run that would not consume notibable time.
081351 -
081352 -     [On 031129 tried this idea and it turned out not to work as well
081353 -     as hoped. ref SDS 8 0001
081355 -  ..
081356 - Another idea is to use /min instead of /b in the set command for
081357 - environment variables.
081358 -
081359 -     This idea turned out not to work for other situations, so had to
081360 -     abandon using /min.
081361 -
081362 -
081363 -
081364 -
081366 -  ..
0814 -
0815 -
0816 - Progress
0817 -
081701 - 0702051 SDS Format Flexible Structure for Control Fields
081702 -
081703 - Follow up ref SDS 6 6R6F, ref SDS 4 BN3O.
081704 -
081705 - As a result of problems converting SDS records to a new subject index
081706 - account structure, reported on 031125, ref SDS 7 WB5N, had to do a
081707 - number of things that fine tune formatting improvements for the F2
081708 - save operation reported on 031124. ref SDS 6 0001
081710 -  ..
081711 - Worked on consistent formatting for j flags and underlined headlines
081712 - when anchors are added for various use cases that surfaced when
081713 - running macro 142 that calls 0702051 to update data bases for SDS
081714 - references, reported on 031125. ref SDS 7 0001  Eventally got a broad
081715 - cross section of 1000 records to successfully run all format control
081716 - support in 0702051. ref SDS 7 GT5X
081717 -
081718 -
081719 -
081720 -
081721 -
081722 -
081723 -
0818 -

SUBJECTS
Save F2 SDS Records Out of Memory Updating Reference Pointers Solved
0702051 F2 Save Highlight J Headline Flags Added Automatically by F2
Start 0702051 F2 Save Fix Out of Memory Updating Reference Pointers S
0702051 F2 Save SDS Records Out of Memory Updating Reference Pointers
Reference Pointers Memory Failed F2 Save Updated in 0702051 Solved wi
0702051 Save F2 Macro 142 in 000008 Process Reference Data Bases in A

1308 -
130901 -  ..
130902 - Reference Data Bases Processed in Separate Memory Segments
130903 - 0702051 F2 Save Windows Start Nested Application Reference Pointers
130904 -
130905 - On 031125 encountered a new memory issue during work to convert SDS
130906 - records and data base files from a legacy system developed for the
130907 - "sales points" account, to the new system of naturally chained
130908 - segmented structures. ref SDS 7 ZW5I
130910 -  ..
130911 - Line 1500, ref OF 5 1937, -entry 142
130912 -
130913 -    dose /c c:\sd\01\0702051.bat
130914 -
130915 -       Windows start command is used to expand memory for saving SDS
130916 -       records called in 000008 under macro 142.
130918 -  ..
130919 - Design of reference processing reported on 010510 aims to maximize
130920 - speed and reduce memory. ref SDS 1 0001  The record today indicates
130921 - the design is effective on SDS records that are near the limits of DOS
130922 - mmeory by requiring only 30% of capacity to execute the save command
130923 - for updating data bases that contain SDS references. ref SDS 7 ZW6I
130924 - However, since processing failed on 031125 using only 30% of memory,
130925 - ref SDS 0 EG8G, this suggests the process design is fragmenting memory
130926 - and this condition of fragemented memory is overwhelmed when a large
130927 - block of additional data is read into memory for updating a weekly
130928 - reference data base file.
130930 -  ..
130931 - After changes were made today, shown below, testing on 1000 files ran
130932 - consecutivley for 15 minutes, reported on 031125. ref SDS 7 ZW5I  This
130933 - suggests the change today is effective.  However, as noted above, the
130934 - process is very slow and the screen is very unstable. ref SDS 0 HH9N
130935 -
130936 -        [On 031203 reported completion of work revising design of F2
130937 -        save process in 0702051 to increase speed and maintain screen
130938 -        stability, while retaining memory capacity developed today.
130939 -        ref SDS 9 0001
130941 -  ..
130942 - Line 1670, ref OF 1 XQ4O, -label ppntrs in 0702051
130943 -
130944 -    This is where processing begins to manage SDS data base references
130945 -    for 01, 03, 04 and 05 lines in SDS records.
130946 -
130947 -    The new design will configure the code in 0702051 for processing
130948 -    references, then save the code in 0702051 to a temporary file that
130949 -    is called to process only references in a separate memory segment.
130950 -    The primary segment will then extract the references and save the
130951 -    result to another temporary filename.  Start will execute the data
130952 -    file with the temporary processor configured in 0702051.  This
130953 -    requires a call at the beginning of 0702051 to the location that
130954 -    processes the data base.  After this is completed, the temporary
130955 -    processor will purge the data base and quit, returing control back
130956 -    to the main processor in 0702051.  The code in 0702051 will then
130957 -    empty the content of the file and read in the original record to
130958 -    select the next group of reference lines, then run the same cycle
130959 -    for updating the data base.  This results in each of the four (4)
130960 -    data base types being processed in a fresh memory segment, and so
130961 -    should help relieve the problem reported on 031125, per above.
130962 -    ref SDS 0 EG8G
130964 -  ..
130965 - Line 500, ref OF 3 XP3F, -label 165g1 in 0702053
130966 -
130967 -    -label 165g1
130968 -    save stufqq1
130969 -
130970 -        After setting up 0702051 to update a weekly Reference pointer
130971 -        data base file, save the settings to temporary file stufqq1 to
130972 -        be called as a processor in a Windows Start memory session.
130974 -         ..
130975 -        Another idea is to create a separate procesor which would be
130976 -        only about 1K, rather than save and load 0702052 each pass
130977 -        which is 10K.  The code would have to be modified to load the
130978 -        new processor each time after references are developed and
130979 -        saved to the disk, then purged to read back the pointer file,
130980 -        and then save it to the disk again.  This would be faster
130981 -        because would be reading and writing 1K each pass rather than
130982 -        10K.
130983 -
130985 -  ..
130986 - Line 2110, ref OF 3 2T4K, -label dBResto in 0702053
130987 -
130988 -    -label dBResto
130989 -    e stufq
130990 -    -if @0 != 989 save stufqq2
130991 -    setcnt 0 0
130992 -    dose /c c:\sd\01\0702051a.bat
130993 -
130994 -        e stufq returns to SDS record where original content has been
130995 -        saved to the disk, and at this point everything has been
130996 -        deleted in the current memory space for the record except for
130997 -        Reference data base lines.  -if @0 != 989 there are Reference
130998 -        data base lines to process, so save to a different temporary
130999 -        file for processing with dose /c c:\sd\01\0702051a.bat that
131000 -        calls another Windows Start command memory session.  -if @0 =
131001 -        989 the record is not saved.  The new batch file developed
131002 -        today for this op, 0702051a.bat, deletes the temporary data
131003 -        base transfer file each pass, so when the process runs in the
131004 -        next step, if the file is blank, then the process will delete
131005 -        any lines in the data base for this record to conform to the
131006 -        condition that there are no data base lines for this record.
131008 -     ..
131009 -    -goto blnKr
131010 -
131011 -        After processing to update the Reference data base files, jump
131012 -        to process the next data base type.
131013 -
131015 -  ..
131016 - Line 2150, ref OF 1 ZC9N, -label rfNcS in 0702051
131017 -
131018 -    -label rfNcS
131019 -    errorcnt 0 0
131020 -    bot
131021 -    linecnt 1 0
131022 -    immed /
131023 -
131024 -        -label rfNcS is added, otherwise this code remains the same,
131025 -        except now it is run from the 3rd session of memory using a 2nd
131026 -        call to the Windows Start command per above. ref SDS 0 P45T
131027 -
131028 -     -label vbht
131029 -    get D:\sd\08\UUUUU\01\yy\zz
131030 -    loc_cur 3 1
131031 -    -if @0 > 0 up
131032 -    setlen 111 0
131033 -    -if @111 > 0 -goto notBL
131034 -    quit
131035 -
131036 -
131037 -
131038 -
131039 -
131040 -
131041 -
131042 -
131043 -
131044 -
131045 -
131046 -
131047 -
1311 -