CONTACTS 

SUBJECTS
Segment Index into Smaller Indexes that Fit SDS Memory Limi  tions
Segmented Subject Indexes Sort that Fit Medit Memory Limitations Use
Sales Points Converted Legacy Segmented Indexes to New Standard Chain

0605 -
0605 -    ..
0606 - Summary/Objective
0607 -
060701 - Follow up ref SDS 6 0000, ref SDS 5 0000.
060702 -
060703 - Converted sales points subject indexes into chained and segmented
060704 - indexes so there is a single method in SDS for Context Management
060705 - using organic structures.  Segmenting the indexes themselves was
060706 - accomplished fairly quickly in an hour or so.  Converting the records
060707 - to apply the new assignments took a lot longer to develop a method and
060708 - to make other changes to legacy SDS records that support the new
060709 - features in the SDS F2 Save function.  This led to additional
060710 - improvements in the code for "save." ref SDS 0 WB5N  Once this was
060711 - accomplished, the actual conversion took 15 minutes to run.  Testing
060712 - indicates conversion was successful. ref SDS 0 MX4N
060713 -
060714 -       [On 031128 modified 0702051 to process reference data base files
060715 -       in a tertiary memory level for the F2 save function.  After this
060716 -       change, the conversion of about 1000 records ran successfully in
060717 -       about 15 minutes. ref SDS 9 EG8G
060718 -
060719 -
060720 -
060721 -
060723 -  ..
0608 -
0609 -
0610 - Background
0611 -
061101 - On 031114 improved subject index so that when an index is sorted to
061102 - find a subject, a form of virtual memory is used for scrolling through
061103 - large indexes.  The key method of adding to an index is to create
061104 - segments and chain them together, developed on 030810. ref SDS 5 0001
061105 - At that time on 030810 the record shows that one class of subjects for
061106 - sales points is handled differently, because there was a legacy system
061107 - developed years earlier on 980107 to solve the memory problem.
061108 - ref SDS 2 NQ32, as related in the record on 030810. ref SDS 5 QG7I
061110 -  ..
061111 - Today, I want to convert the sales point account to use the new
061112 - method of chained indexes, because this provides unlimited capacity.
061114 -  ..
061115 - There are several steps....
061116 -
061117 -    1.  Convert indexes....
061118 -
061119 -        Combine index 1d and 1e into 1c and then segment 1c using
061120 -        program system.
061122 -         ..
061123 -        This simply entails opening 1d and saving to a temp file, say
061124 -        1d on the local directory, then opening the 1c subject index.
061126 -             ..
061127 -        a.  Segment 1c using the standard menu call developed on
061128 -            030810. ref SDS 5 Q15J  Make the last segment small enough
061129 -            so the 1d file can be read in.  Since 1d is only about 50K
061130 -            this is not an issue.  Made 1c into two segments.
061132 -             ..
061133 -        b.  Set titles off and use the get command to read 1d below
061134 -            the last segment in 1c.  Use the change command to change
061135 -            the column for "1D" into "1C."  Save the index.  At this
061136 -            time, "1D" has become part of 1C again.
061138 -             ..
061139 -        c.  Segment the new 1C segment which now includes what was
061140 -            previously 1D into smaller parts.  Made this into 3 new
061141 -            segments, so now 1c has 5 segments.
061143 -             ..
061144 -        d.  Repeat step 2 to bring the prior 1e index into the last
061145 -            segment of 1c.  Repeat step 2 to segment this last segment
061146 -            of 1c again.  Made another 3 new segments, so that 1C is
061147 -            not 8 segements.
061149 -             ..
061150 -        e.  Remove the priod indexes for 1D and 1E, because everything
061151 -            is not in 1C.
061153 -         ..
061154 -    2.  Open all SDS records and change 1d and 1e in subject control
061155 -        fields to 1c.  Below, converted an existing macro 0810 for this
061156 -        job. ref SDS 0 PR5H
061158 -         ..
061159 -        At this point, we could call macro 142 calling 0702051 to
061160 -        execute the standard SDS record (F2) save function and this
061161 -        would update all of the pointers in the entire data base.
061163 -         ..
061164 -        We have done things like this previously.....
061165 -
061166 -        a.  01 04 08050....................... 900131, ref SDS 1 0001
061167 -
061168 -              This is a generic converter for various
061169 -              tasks that do not really apply to current
061170 -              requirement.
061172 -             ..
061173 -        b.  01 04 08100....................... 989107, ref OF 3 5Z5H
061174 -
061175 -              This is closer to what we need, because it was used to
061176 -              convert the stuff from 1C to 1D and 1E, so should be
061177 -              helpful to convert back.... maybe. ref OF 3 JD7M
061178 -
061179 -              Wound up modifying this processor, per below. ref SDS 0
061180 -              PR5H
061182 -             ..
061183 -        c.  01 04 08100....................... 989107, ref OF 3 5Z5H
061184 -
061185 -              This one converted the pointers.
061186 -
061187 -              Wound up using the F2 save method, discussed above,
061188 -              ref SDS 0 4W3O, incorporated into macro file 0810, per
061189 -              below.
061190 -
061192 -         ..
061193 -    3.  Modify the sort op in 03501 to support change back from 1d and
061194 -        1e to all 1c, which means a single system.  On 030810 this
061195 -        issue is reviewed in connection with creating a new "virtual
061196 -        memory" method for viewing sorted organic structures.
061197 -        ref SDS 5 U87K
061198 -
061199 -            [On 031128 updated code in 03501 to eliminate special
061200 -            support for sorting sales point indexes. ref SDS 10 736N
061201 -
061202 -
061204 -  ..
061205 - 0810  Convert Subjects for ...1D and 1E to All 1C Accounts
061206 -
061207 - Gary called and reported having converted SDS records and reference
061208 - pointer data bases.  His records have been out of sync, so a little
061209 - leary about copying things from his computer that pertain to my SDS
061210 - records.
061212 -  ..
061213 - Decided this evening to try following method...
061215 -  ..
061216 - Line 70, ref OF 3 U78K, -label lp1 in 0810
061217 -
061218 -    Modified 0810 to apply Windows start command similar to the system
061219 -    developed for 070302 on 031124. ref SDS 7 XA6O
061220 -
061221 -    Ran a summary subject report on........
061222 -
061223 -                 03 00050 01 05 1D
061224 -
061225 -    ...and...
061227 -                  ..
061228 -                 03 00050 01 05 1E
061229 -
061230 -    ....for the past 20 years.
061232 -     ..
061233 -    Had to do this in several lifts, because the list is too big to fit
061234 -    in memory for the 20 year period.  This took about 10 minutes,
061235 -    mostly to set up separate report specs.
061237 -     ..
061238 -    Opened each report, and shifted SDS record spec from column 171 to
061239 -    column 1.  After this shift, the files were much smaller and so
061240 -    could be joined into a single file that Medit can handle.
061242 -     ..
061243 -    The file was then sorted and duplicate keys were omitted, so we
061244 -    wind up with a single list for converting all SDS records with
061245 -    either 1D or 1E into 1C.  There were a total of...
061246 -
061247 -                          976 records
061249 -     ..
061250 -    Used Opt tech sort as follows...
061251 -
061252 -        o f1 g1 /s(1,13,c,a) dupout(1,13)
061253 -
061254 -    ...based on research reported 010301. ref SDS 3 ER3K
061255 -
061256 -
061258 -  ..
061259 - Line 70, ref OF 3 U78K, -label lp1 in 0810
061260 -
061261 -    -label lp1
061262 -    loc_cur 4 1
061263 -    linecnt 1 0
061264 -    -if @0 > 0 -goto end
061265 -    macro 891
061266 -    up
061267 -
061268 -        This is called simply from the command line...
061269 -
061270 -                     CMD>@c:\sd\03\0810
061272 -         ..
061273 -        This is a simple loop for macro 891 to read a file spec, the
061274 -        code then scrolls the file up ready to ready the next one.
061275 -        linecnt 1 0 is simply to generate an error to indicate the
061276 -        bottom of the list has been reached, so the op is over.
061277 -
061278 -
061279 -
061281 -     ..
061282 -    Start Memory Session Process Each Record Separately
061283 -
061285 -  ..
061286 - Line 80, ref OF 3 TH9N, -label lp1 in 0810
061287 -
061288 -    e tje4h
061289 -    -if @60 = 1 -goto open
061290 -    macro 301
061291 -    setcnt 60 1
061292 -
061293 -        Create temporary batch file, similar to work on 021121 to
061294 -        modify macro file 070301 for processing records to convert and
061295 -        action item report into an SDS record to review progress.
061296 -        ref SDS 7 XA6O  -if @60 = 1 the file is already open, so skip
061297 -        set up.  macro 301 empties the file for setup if there is
061298 -        already something there for some reason or another.
061300 -     ..
061301 -    immed 4a
061302 -    ins_text !@echo off!
061303 -    rel_cur 1 0
061304 -    ins_text !c:!
061305 -    rel_cur 1 0
061306 -    ins_text !cd\sd\10!
061307 -    rel_cur 1 0
061308 -    ins_text !%AIR% c:\sd\01\e d:\sd\08\uuuuu\!
061309 -
061310 -        set command in sds.bat configures "air" for action item report
061311 -        so works the same for this task.
061312 -        Would like to put this set command in this processor, but
061313 -        seems that multiple passes of the set command returns an
061314 -        error message "Out of environment space."
061315 -
061316 -
061317 -
061318 -
061320 -      ..
061321 -
061322 -                             Enter file spec
061323 -    -label open
061324 -    loc_cur 0 27
061326 -     ..
061327 -    macro 641
061328 -
061329 -                                    Get globals
061330 -    macro 91
061331 -                                    Enter file user ID and dir for the record
061332 -    macro 88
061334 -     ..
061335 -    rel_cur 0 1
061336 -    ins_text !c:\sd\03\0810 #2!
061337 -
061338 -    -if @61 = 1 -goto save
061339 -    setcnt 61 1
061341 -     ..
061342 -    loc_cur 0 1
061344 -     ..
061345 -    rel_cur 1 0
061346 -    ins_text !exit!
061347 -    rel_cur -1 0
061348 -
061349 -    -label save
061350 -    save c:\sd\10\stufqe.bat
061351 -    **
061352 -    **
061353 -    save c:\10\tst1
061355 -                                ..
061356 -                               Return to the SDS record for AI review
061358 -     ..
061359 -    ecur 1
061361 -      ..
061362 -
061363 -                          Get all of the action items in this record.
061365 -     ..
061366 -    dose /c c:\sd\10\stufqe.bat
061367 -
061368 -    -goto lp1
061369 -
061370 -
061372 -     ..
061373 -    Change SDS Records in 2nd Memory Session
061374 -
061375 -
061376 - Line 140, ref OF 3 TG7L, -label strtop in 0810
061377 -
061378 -    -label strtop
061379 -    top
061380 -    z 53 69
061381 -    errorcnt 0 0
061382 -    c /03 00050 01 05 1D/03 00050 01 05 1C/* 1
061383 -    c /03 00050 01 05 1E/03 00050 01 05 1C/* 1
061385 -         ..
061386 -        Added this code for a 2nd memory session to change records in
061387 -        the list were 1D and 1E subjects occur.
061388 -
061389 -
061391 -     ..
061392 -    Reference Pointer Data Base Updated with Standard F2 Save
061393 -
061394 -
061395 - Line 180, ref OF 3 CZ9H, -label strtop in 0810
061396 -
061397 -    setcnt 47 2
061398 -
061399 -       Set up standard SDS environment after control fields are
061400 -       modified.
061402 -     ..
061403 -    setgbl 275 48
061404 -    setgbl 276 48
061405 -    setgbl 277 49
061406 -    setgbl 278 48
061407 -    setgbl 279 49
061408 -    setcnt 271 2000
061409 -    macro 142
061410 -    quit
061411 -
061412 -       Set up global User ID for these records; setcnt 271 2000 makes
061413 -       all records archived diaries. macro 142 runs standard F2 save op
061414 -       calling macro file 0702051 to update reference pointers, per
061415 -       above planning. ref SDS 0 4W3O
061416 -
061418 -     ..
061419 -    Ran into a snag because some of the older records do no support all
061420 -    of the features in the save op under macro 142 and 0702051.  For
061421 -    example, on 031124 added new support in 0702051 for consistant
061422 -    formatting. ref SDS 8 0001  Was able to determine records that
061423 -    cause the problem by examining stufqe.bat since it is saved each
061424 -    pass and has the last file processed.
061425 -
061426 -       [On 031128 improved format control and management in 0702051 to
061427 -       handle wider range of use cases. ref SDS 9 6R6F
061429 -  ..
061430 - Seems to run and then stop on...
061431 -
061432 -            011003............................... ref SDS 4 0001
061433 -
061434 -     This one is crashing with out of memory message in 0702051,
061435 -     suggesting the lastest changes may have tipped the balance on
061436 -     this file.
061437 -
061438 -         [On 031128 modified 0702051 to process reference data base
061439 -         files in a tertiary memory level for the F2 save function.
061440 -         After this change, the conversion of about 1000 records ran
061441 -         successfully in about 15 minutes. ref SDS 9 EG8G
061443 -  ..
061444 - Line 1760, ref OF 1 035H, -label nvMnd1 in 0702051 about 50 lines
061445 - below
061446 -
061447 -    Tried debug here.
061448 -
061449 -    Getting out of memory message when running macro 142 that calls
061450 -    0702051 under 0810, but there is no error message when the same
061451 -    call is made in an SDS record using the standard F2 function.
061453 -     ..
061454 -    In both settings the Medit free command reports....
061455 -
061456 -                            9672
061457 -
061458 -
061459 -    ...which suggests about 70% of the memory is still available, based
061460 -    on testing that indicates failure when free nears 30K.
061462 -     ..
061463 -    This presents an anomoly.  Why is there an out of memory message
061464 -    when free is only 9672, and why does one situation fail and other
061465 -    runs?
061467 -     ..
061468 -    The difference between the settings is that the one that fails is a
061469 -    2d level memory session using the Windows Start command, and the
061470 -    one that runs is a 1st level memory session.  This suggests
061471 -    something is occurring to memory that is not evident from using the
061472 -    Windows Start command.
061474 -        ..
061475 -       [On 031128 later modified 0702051 to use a separate memory
061476 -       segment for processing reference data base files.  After this
061477 -       change was made, the conversion of about 1000 records ran
061478 -       successfully in about 15 minutes. ref SDS 9 EG8G
061479 -
061480 -
061481 -
061482 -
061483 -
061484 -
061485 -
061486 -
061487 -
061488 -
061489 -
061490 -
061491 -
061492 -
0615 -