THE WELCH COMPANY
440 Davis Court #1602
San Francisco, CA 94111-2496
415 781 5700
rodwelch@pacbell.net


S U M M A R Y


DIARY: February 14, 2004 12:08 PM Saturday; Rod Welch

SDS added memory level to convert records into HTML for Internet.

1...Summary/Objective
2...070405 Add Another Segment for Multi-level Virtual Memory Processing
3...Link Processing in 2nd Level Setup for 3rd Memory Level 1800 Lines
4...Link Processing Segments Determined by Counter 143 and 43
5...Ctrl D Identify End of Record to Determine End of Linking Process
6...Stufqv Save Temp for Header, Summary and Content Lines
7...References for Link Processing Identified by Counters 40 and 41
8...Header and Summary Saved Separately, Reference Lines Determined
9...References Configure Get Command in 070405 to Process Links
10...Calculate Number of Lines for Reference Field Subroutine
11...Get Command Configured for Subroutine to Read in Referneces
12...Get Command Configure in 070405 Setup 3rd Level Processing for Links
13...3rd Level Processing Setup Get Command in 070405
14...Get Command Configure in 070405 Setup 4th Level Processing for Links
15...4th Level for Secondary Processing Links if More than 1800 Lines
16...Links Convert into HTML in 3rd Memory Level Process 1800 Lines
17...Links Main Process Convert into HTML
18...Anchors, Explicit Links Compplete Processing Level 3 Memory Segment
19...Level 3 Virtual Memory Segment for Anchors Explicit Links
20...Links Determine Multi-level Processing Required, Launch Level 4
21...Level 4 Launch Sub-process to Complete Convert Links into HTML
22...References Remove from Contene with Links Converted
23...End of Record All Links Converted into HTML
24...Links Complete Process to Convert into HTML, Next Step Process Anchors
25...Level 4 Virtual Memory Segment for Sub-processing Over 1800 Links
26...References Subroutine Get from Stufq Each Pass to Process Links
27...070405.bat Modify Start Op for Multi-level Processing for Links
28...070405.bat Launch New Memory Segment to Process Links
29...070405a.bat 4th Level Memory Segment Process Links Above 1800 Lines


..............
Click here to comment!

CONTACTS 

SUBJECTS
Programs Launch from Within SDS Based on File Extension to Microsoft Wi
3-levels 070405 Multi-level Processing Save Memory Avoid Fragments Us
Anchors Convert into HTML 070405 Complete Processing 3-levels 070405
SDS Records 070405 3-levels 070405 Multi-level Processing Save Memory
070405 Convert SDS Record to HTML Failing on Large Record
Windows Start Command Called by 0608 to Call 0122 for Converting Docu
Added 3rd Memory Segment to have 4 Processing Levels so that One is

1609 -
1609 -    ..
1610 - Summary/Objective
1611 -
161101 - Follow up ref SDS 6 0000, ref SDS 4 0000.
161102 -
161103 - Improved virtual memory support to increase capacity by about 50%
161104 - using 070405 to convert an SDS record into HTML for display in web
161105 - browsers on the Internet.  Review shows the problem of fragmented
161106 - memory causing program failure when slightly over half of allocated
161107 - memory is being used.  Experience developing similar solutions on
161108 - other parts of the program helped expedite developing code for adding
161109 - another segment to multi-level memory processing in 070405.
161110 - ref SDS 0 WM6G
161111 -
161112 -
161113 -
161114 -
161115 -
161117 -  ..
1612 -
1613 -
1614 - Evaluation
1615 -
161501 - This is a fairly major improvement using the virtual memory model
161502 - developed for the 0702051 save function on 031203. ref SDS 5 WH6M
161504 -  ..
161505 - Testing on the most difficult records indicates the new code developed
161506 - today (and continuing over the next 4 weeks) is solving the problem of
161507 - fragmented memory by using a "virtual memory" model based on the
161508 - Windows Start command to open fresh levels of memory segments.
161509 - Linking is the most problematic memory process because the length of
161510 - links on a line is increased by about 500%. In a densely linked
161511 - record, Medit fails.  The solution seems to work by performing links
161512 - in a separate memory segment, ref SDS 0 HY55, and to further segment
161513 - the process into another level when there are more than 1800 lines in
161514 - a record. ref SDS 0 G28O
161516 -  ..
161517 - Testing the following records passed....
161518 -
161519 -                      Words  Lines Links  Size
161520 -
161521 -        890809........8410...2831...332....167K...... ref SDS 1 0001
161522 -        931130.......10525...2973...406....168K...... ref SDS 2 0001
161523 -        010725........9396...2476...310....160K...... ref SDS 3 0001
161524 -        040102.......14103...3005...431....192K...... ref SDS 7 0001
161525 -
161526 -
161527 -
161528 -
161529 -
161530 -
161532 -  ..
1616 -
1617 -
1618 - Problem
1619 -
161901 - This afternoon noticed an error in the record on 040201 that reviews
161902 - Gary's work using SDS for the eight (8) steps to perform Communication
161903 - Metrics at Aerospace company. ref SDS 7 JY5K   Made the correction and called
161904 - 070405 to generate a file for display on the Internet.  The program
161905 - did not complete processing, but hung, indicating a problem in 070405.
161906 - Per evaluation above, the record on 040102 seems to be the biggest
161907 - record in the system. ref SDS 0 RP4L
161909 -  ..
161910 - Review shows the following...
161911 -
161912 -    1.  Memory fails at very end of the process when all of the changes
161913 -        to the record are written to the disk; the code first executes
161914 -        a change command converting the local directory address to an
161915 -        Internet address. ref OF 1 4260
161917 -         ..
161918 -    2.  When memory fails Medit diagnostics show 21K, which is only
161919 -        about 66% of capacity.  This indicates that failure occurs
161920 -        because memory has been fragmented by preceeding processes, and
161921 -        so there are not enough spaces left that are big enough to
161922 -        change the address for all of the links in the record.
161923 -        Therefore, we need a 3rd memory level segment so that the 2nd
161924 -        level can be allocated solely to processing links. ref SDS 0
161925 -        4N8F  We should be able to use a code scheme similar to the
161926 -        design developed on 031203 for 0702051 to process control
161927 -        fields saving large records. ref SDS 5 WH6M
161929 -         ..
161930 -    3.  Memory, also, fails during the process that converts links into
161931 -        HTML code on large records with a lot of links.  The link
161932 -        conversion process causes severe memory fragmenting by more
161933 -        than doubling the line length on a large percentage of lines.
161934 -        Therefore, memory fails converting large SDS records, like for
161935 -        040102, which has about 420 links. ref SDS 7 0001  We need a
161936 -        new loop in a fresh memory sub-segment within the new level 2
161937 -        cycle that processes links, under para 2, ref SDS 0 HY4G, so
161938 -        that one cycle in one memory segment processes no more than
161939 -        about 1800 lines and less than 400 links in a single lift. 1800
161940 -        lines is based on experience showing the code can handle about
161941 -        2900 lines with about 400 links. ref SDS 0 VF4I  We also need
161942 -        to design a floating record size, so that the sub-level process
161943 -        for linking within level 2, only processes 1 or 2 lines.
161944 -        ref SDS 0 VF4M
161945 -
161946 -
161947 -
161948 -
161949 -
161951 -  ..
1620 -
1621 -
1622 - Solution
1623 -
162301 - 070405  Add Another Segment for Multi-level Virtual Memory Processing
162302 -
162303 - Follow up ref SDS 4 DF5I.
162304 -
162306 -  ..
162307 - Line 30, ref OF 1 MI4M, in 070405 at top of file
162308 -
162309 -    -goto level2............................... ref SDS 0 4P4F
162310 -
162311 -        2nd processing level is now devoted exclusively to convert
162312 -        links into HTML, and it now has a secondary memory sub-level
162313 -        under -label level4, ref SDS 0 G28O, per planning above.
162314 -        ref SDS 0 WX5J
162315 -
162317 -  ..
162318 - Line 40, ref OF 1 I55K, in 070405 at the top of the file.
162319 -
162320 -    -goto level3.............................. ref SDS 0 KT92
162321 -
162322 -        Added new processing level 3 in 070405 to solve memory problem,
162323 -        per above. ref SDS 0 HY4G  This complements level 2 developed
162324 -        on 030202, ref SDS 4 DF5I; it converts anchors and creates
162325 -        explicit links, then completes processing by saving the entire
162326 -        file to the local web directory, wl, then executes a global
162327 -        change to all the links for the Internet address and saves
162328 -        these changes to the transfer directory, wt, and then ends
162329 -        processing to return control to the User.
162330 -
162331 -
162333 -  ..
162334 - Line 50, ref OF 1 RP5O, in 070405 at top of the file
162335 -
162336 -    -goto level4.............................. ref SDS 0 PQWV
162337 -
162338 -        New sub-processing level 4 in 070405 called under level 2 to
162339 -        convert links in large records with more than 1800 lines.
162340 -        ref SDS 0 KS6H, per plannng above. ref SDS 0 WX5J  A new batch
162341 -        file 070405a.bat, ref SDS 0 WW3I, solves the memory problem,
162342 -        per above. ref SDS 0 WX5J
162343 -
162344 -
162346 -  ..
162347 - Link Processing in 2nd Level Setup for 3rd Memory Level 1800 Lines
162348 -
162349 - Added code to save the header and summary separately, and also the
162350 - References separately on the disk, during processing of links.  This
162351 - saves some memory.  Developed system to read in the References and to
162352 - calculate the line numbers to read from the original record in stufq
162353 - only a portion of SDS record content for processing links, when large
162354 - files have more than 1800 lines.  This design permits processing
162355 - smaller files in memory, that are less conducive to failure due to
162356 - memory fragments, because there is more capacity available relative to
162357 - the smaller segments being processed.  The pieces are then copied back
162358 - together at the operating system level on the disk by the batch files
162359 - for final processing by 070405. ref SDS 0 WV5M
162361 -  ..
162362 - The 1800 line figure is experimental; it reflects experience today
162363 - showing the record on 040102 was processed successfully with the
162364 - existing code, until a few more lines, forward links and other links
162365 - were added.  The current file has about 3005 lines with 431 links,
162366 - ref SDS 0 RP4L, suggesting the existing code can handle something just
162367 - less than those numbers, and so selecting 1800 lines seems likely to
162368 - provide a margin of safety in the event a record occurs that is more
162369 - densely populated with links than the record on 040102. ref SDS 7 0001
162371 -  ..
162372 - Also, try to provide a floating virtual memory segment so that the
162373 - secondary level for processing links does not wind up with only 1 or 2
162374 - lines. ref SDS 0 VF4M
162375 -
162376 -
162378 -  ..
162379 - Link Processing Segments Determined by Counter 143 and 43
162380 - Ctrl D Identify End of Record to Determine End of Linking Process
162381 -
162382 - Line 2350, ref OF 1 HW6I, -label cites in 070405
162383 -
162384 -    -label cites
162385 -    ic 4.........temp end of record..see -label distr.. ref SDS 0 4R4G
162386 -    save stufq
162387 -    linecnt 144 0
162388 -    setcnt 143 1800
162389 -    -if @144 > 1600 -if @144 < 2200 setcnt 143 1200
162390 -    setgbl 143 @143
162391 -
162392 -        Processing is at the last line in the file, after formatting
162393 -        for identification all forward links.  ic 4 creates a temporary
162394 -        end of record mark to tell the code when to open another memory
162395 -        segment to complete processing the rest of the lines in the
162396 -        record for converting links into HTML -- this is a key part of
162397 -        virtual memory management;  linecnt 144 0 gets the number of
162398 -        lines to process, which is actually less, after subtracting the
162399 -        summary header and Reference lines, however, this calculation
162400 -        is done at this point in order to have a constant method for
162401 -        reading contiguous memory segments from the original SDS record
162402 -        as processed up to this point, which is in the stufq file.
162403 -
162404 -
162405 -
162407 -  ..
162408 - Stufqv Save Temp for Header, Summary and Content Lines
162409 - References for Link Processing Identified by Counters 40 and 41
162410 - Header and Summary Saved Separately, Reference Lines Determined
162411 -
162413 -  ..
162414 - Line 2410, ref OF 1 DX8H, -label cites in 070405 about 60 lines below
162415 -
162416 -    top
162417 -    z 1 3
162418 -    l /03 /
162419 -    linecnt 40 0
162420 -    z 1 1
162421 -    l / /
162422 -    linecnt 41 0
162423 -    line @40
162424 -    down
162425 -    immed a
162426 -    rel_cur 2 0
162427 -    immed 9999d
162428 -    save stufqv................................ ref SDS 0 LT5M
162429 -
162430 -        After calcuating how many lines to process, either the default
162431 -        1800 or the alternative 1200, ref SDS 0 VF4M, top moves the
162432 -        file into position to get line numbers for the Header and
162433 -        Summary and for References, needed in the 3rd memory segment to
162434 -        process links. z 1 3 l /03/ finds the bottom of the Header and
162435 -        Summary, which is the top line of References.  linecnt 40 0
162436 -        identifies this location in the original SDS record which is
162437 -        now stufq.  z 1 1 adn l / / finds the space at the bottom of
162438 -        References linecnt 41 0 identifies this location for the bottom
162439 -        of References and the top of the record content that has the
162440 -        links to be processed in this operation.  Now that is
162441 -        information is captured, line @40 0 returns to the bottom of
162442 -        the Header and Summary lines.  Add a blank immed a needed to
162443 -        format the transition from Contacts to References in the final
162444 -        rendering, and then immed 9999d deletes all the lines in the
162445 -        file so the Header and Summary lines can be saved as a separate
162446 -        file, stufqv, that is on the disk until all of the links have
162447 -        been processed, and then it will be copied together by
162448 -        070405.bat with the content for final processing.
162449 -
162450 -
162451 -
162453 -  ..
162454 - References Configure Get Command in 070405 to Process Links
162455 - Calculate Number of Lines for Reference Field Subroutine
162456 - Get Command Configured for Subroutine to Read in Referneces
162457 -
162459 -  ..
162460 - Line 2430, ref OF 1 TP4H, -label cites in 070405 about 80 lines below
162461 -
162462 -    setgbl 40 @40
162463 -    setgbl 41 @41
162464 -    e c:\sd\03\070405
162465 -    purge stufq
162466 -    line..........................&& *%lNt......... ref SDS 0 PTXT
162467 -    getgbl 40 40
162468 -    getgbl 41 41
162469 -    loc_cur 4 9
162470 -    ins_text /         /
162471 -    inscnt 40 0
162472 -    rel_cur 0 1
162473 -    setcnt 42 @41
162474 -    subcnt 42 @40
162475 -    inscnt 42 0
162476 -
162477 -        glogals 40 and 41 are set from capturing line numbers for
162478 -        References in stufq, per above. ref SDS 0 VG5O  e 070405 opens
162479 -        the processor; purge stufq removes SDS record from memory to
162480 -        get ready for launching the 3rd memory level to process links;
162481 -        first, configure get command in 070405 for processing in the
162482 -        3rd memory level to read in the References in multiple passes
162483 -        if needed.  get globals just saved in stufq. line ....&& *%lNt
162484 -        moves to subroutine for References to post get command. loc_cur
162485 -        4 9 positions cursor to enter 1st param for the get command,
162486 -        which is the 1st line to begin reading from stufq.  ins_text
162487 -        /............./ clears any prior entry. inscnt 40 0 enters the
162488 -        line number for the top of the Reference field in stufq.
162489 -        rel_cur 0 1 positions to enter the number of lines to read.
162490 -        The number of lines to read is calculated by the difference
162491 -        between counter 41 that has the line number at the bottom of
162492 -        the Reference field, and counter 40 that has the line number
162493 -        for the top of the Reference field.  Subtracing these numbers
162494 -        yields the number of lines to read for all of the References.
162495 -
162496 -
162497 -
162499 -  ..
162500 - Get Command Configure in 070405 Setup 3rd Level Processing for Links
162501 - 3rd Level Processing Setup Get Command in 070405
162502 -
162503 - Line 2520, ref OF 1 2594, -label cites in 070405 about 190 lines below
162504 -
162505 -    line..........................&& *%lNb........... ref SDS 0 4P4F
162506 -    loc_cur 4 9
162507 -    ins_text /             /
162508 -    inscnt 41 0
162509 -    rel_cur 0 1
162510 -    getgbl 43 143................................ ref SDS 0 VF4M
162511 -    inscnt 43 0
162513 -         ..
162514 -        This step completes configuring 070405 to process links in the
162515 -        3rd memory level, by entering beginning line number for SDS
162516 -        record content to get from stufq, and entering the total number
162517 -        of lines read in, normally 1800.  Position cursor on beginning
162518 -        line number in stufq for record content and ins_text /       /
162519 -        clears any prior data for get command.  inscnt 41 0 enters the
162520 -        line number for the bottom of References, which is the top of
162521 -        content with links to process in stufq, determined above.
162522 -        ref SDS 0 VG5O   getgbl 143 143
162523 -
162524 -
162525 -
162527 -  ..
162528 - Get Command Configure in 070405 Setup 4th Level Processing for Links
162529 - 4th Level for Secondary Processing Links if More than 1800 Lines
162530 -
162531 - Line 2540, ref OF 1 O24N, -label cites in 070405 about 220 lines below
162532 -
162533 -    addcnt 41 @43
162534 -    line..........................&& *%lln.......... ref SDS 0 VF4G
162535 -    loc_cur 4 9
162536 -    ins_text /             /
162537 -    inscnt 41 0
162538 -    save
162539 -    quit
162540 -
162541 -        Calculate beginning line number for get command based on adding
162542 -        number of lines processed in counter 43, ref SDS 0 VF4M, for
162543 -        level 3 to the beginning line number in counter 41. ref SDS 0
162544 -        VG5O  save writes 070405 modified with get command params to
162545 -        disk, so when quit occurs and level 2 processing ends, when the
162546 -        next level is opened in batch file 70405.bat by calling macro
162547 -        070405 again, then 070405 is configured to process links. quit
162548 -        command cycles back to 070405.bat op to do next step of
162549 -        converting links into HTML in a new memory segment.
162550 -
162551 -
162552 -
162554 -  ..
162555 - Links Convert into HTML in 3rd Memory Level Process 1800 Lines
162556 -
162557 -
162558 - Line 2570, ref OF 1 AT5O, -label level2 in 070405
162559 -
162560 -    -label level2............................. ref SDS 0 4N8F
162561 -    -gosub rfnCs
162562 -    *%lNb..................................... ref SDS 0 VH5F
162563 -    g stufq n
162565 -         ..
162566 -        This is the 2nd memory segment for 070405 to process links in a
162567 -        fresh memory segement, which is the 3rd memory level starting
162568 -        from the original SDS record. -gosub rfnCs is a new subroutine
162569 -        called to load only References, since the header, Summary,
162570 -        Action item and Contact lines are not needed for converting
162571 -        links into HTML format.  get command is loaded with beginning
162572 -        line number to start reading from file stufq under -label
162573 -        cites, per above. ref SDS 0 VH5F  Number of lines to read in is
162574 -        also determined, normally 1800. ref SDS 0 4P6O
162575 -
162576 -
162577 -
162579 -  ..
162580 - Links Main Process Convert into HTML
162581 -
162582 - Line 2590, ref OF 1 4411, -label lpcn1 in 070405
162583 -
162584 -    -label lpcn1
162585 -    of 0
162586 -    z 8 300
162587 -    errorcnt 0 0
162588 -    l / ref /
162589 -    -if @0 > 0 -goto Distr........................... ref SDS 0 4R4G
162590 -
162591 -        The code is in the 3rd memory segment, which is the 2nd level
162592 -        of processing by 070405, and will convert links into HTML
162593 -        interent addresses.
162595 -         ..
162596 -        This is the main cause of increased memory fragmentation in
162597 -        level 2 processing, because there are a lot of links in SDS
162598 -        records, and each link requires 80 char which increases memory
162599 -        use by about 500% relative to the native SDS link format.  The
162600 -        test record starts at only 15K, but winds up at 23K after this
162601 -        process, and increase of 8K, which is about 50%.
162602 -
162603 -
162604 -
162606 -  ..
162607 - Anchors, Explicit Links Compplete Processing Level 3 Memory Segment
162608 - Level 3 Virtual Memory Segment for Anchors Explicit Links
162609 -
162610 - This code runs after links are converted. It converts anchors and
162611 - creates explicit links, then completes processing and returns control
162612 - to the User.
162614 -  ..
162615 - Line 2960, ref OF 1 GR7I, -label level3 in 070405
162616 -
162617 -    -label level3............................... ref SDS 0 KS6H
162618 -    top
162619 -    errorcnt 0 0
162620 -    z 1 60
162622 -         ..
162623 -        Added new level 3 processing 070405 called at top of record,
162624 -        after the quit command above, ref SDS 0 GA4G, returns to the
162625 -        batch file, and the batch file calls the 3rd level in 070405,
162626 -        per above. ref SDS 0 KS6H   Medit diagnostics show that when
162627 -        the file is opened after the prior step, memory use is down to
162628 -        15K from 23.  This shows 33% reduction in memory requirements
162629 -        compared to the prior design, per above. ref SDS 0 HY4G  Stated
162630 -        another way, the new design is 50% more efficient because the
162631 -        prior design increased memory use from 15K to 23K for the test
162632 -        file.
162634 -         ..
162635 -        The next step converts anchors into HTML, which is a relatively
162636 -        minor increase of 1K in memory use compared to the prior step
162637 -        performed in the prior memory level of converting links into
162638 -        HTML, because links required specifying the Internet address
162639 -        and this is a long string of 80 characters, whereas anchors and
162640 -        the explicit links to aid addressability only add about 40
162641 -        characters.  Since there are many more links than anchors in a
162642 -        record, this increases memory use and fragmentation much more.
162643 -
162644 -
162645 -
162646 -
162648 -  ..
162649 - Links Determine Multi-level Processing Required, Launch Level 4
162650 - Level 4 Launch Sub-process to Complete Convert Links into HTML
162651 -
162652 - Line 3120, ref OF 1 2L4L, -label Distr in 070405
162653 -
162654 -    -label Distr............................. ref SDS 0 WM6G
162655 -    chrcnt 1 0
162656 -    -if @1 = 4 -goto meml3.................... ref SDS 0 4R7L
162657 -    -gosub rfrmv............................... ref SDS 0 4R4O
162658 -    save stufqv1
162659 -    macro 3
162660 -    dose /c c:\sd\01\070405a.bat
162661 -    -goto lvl4q................................... ref SDS 0 4R8J
162662 -
162663 -        At this point, the code has performed an unsuccessful search
162664 -        for another reference to process, and so the cursor is on col 1
162665 -        of the last line in the file. Either....
162666 -
162667 -            a.  All references have been processed and so this op is
162668 -                complete, allowing processing to move to the next op
162669 -                for converting anchors into HTML; or...
162671 -                 ..
162672 -            b.  All lines in the record have not been processed,
162673 -                because the code has configured the record for
162674 -                multi-level procssing for links in another memory
162675 -                segment.
162677 -         ..
162678 -        chrcnt 1 0 reads the position, and if it is a • (ctrl D)
162679 -        symbol, entered before processing links begins, per above,
162680 -        ref SDS 0 VF4M, then the end of the record has been reached,
162681 -        under scenario a.  If the symbol is not present, then launch
162682 -        level 4 processing to complete converting links into HTML. 1st
162683 -        step is to remove references at the top of the file and save
162684 -        the first group of content lines with links converted into HTML
162685 -        to a temporary filename stufqv1.  In the next level, this same
162686 -        code will save the 2nd and final group of content lines with
162687 -        links converted into HTML to temp filename stufqv2, and these
162688 -        two files will joined by batch file 070405a.bat into a single
162689 -        file stufqv1 for final processing launched by 070405.bat.
162690 -
162691 -
162692 -
162693 -
162695 -  ..
162696 - References Remove from Contene with Links Converted
162697 -
162698 - Line 3110, ref OF 1 M56G, -label rfrmv in 070405
162699 -
162700 -    -label rfrmv
162701 -    top
162702 -    immed dd
162703 -    z 1 1
162704 -    l / /
162705 -    immed dd
162706 -    -return
162707 -
162708 -        Subroutine to remove the Reference field when processing links
162709 -        in the 3rd and 4th memory segment levels, if needed, so that
162710 -        the content lines with links converted into HTML code can then
162711 -        be joined together and with the header and Summary lines, to
162712 -        comprise a complete record, for final processing.
162713 -
162714 -
162715 -
162716 -
162718 -  ..
162719 - End of Record All Links Converted into HTML
162720 - Links Complete Process to Convert into HTML, Next Step Process Anchors
162721 -
162722 - Line 3180, ref OF 1 SW7O, -label meml3 in 070405
162723 -
162724 -    -label meml3.............................. ref SDS 0 4R4G
162725 -    ic 32..................................... ref SDS 0 VF4M
162726 -    -gosub rfrmv.............................. ref SDS 0 4R4O
162727 -    -if @290 = 0 save stufqv1
162728 -    -if @290 = 1 save stufqv2
162730 -     ..
162731 -    -label lvl4q.............................. ref SDS 0 4R4G
162732 -    e c:\sd\03\070405
162733 -    purge stufq8
162734 -    quit
162735 -
162736 -        This code is used to complete processing links, either in a
162737 -        single 3rd level processing memory segment, or after completing
162738 -        a sub-level 4th memory segment.  ic 32 removes ctrl d that
162739 -        identifies the end of the record, created when the link
162740 -        conversion process begins above under -label cites. ref SDS 0
162741 -        VF4M counter 290 saves content to either stufqv1 or 2,
162742 -        depending on whether this is a single or double process mode,
162743 -        so that if it is a double process mode, then there are two
162744 -        files, stufqv1 and stufqv2 which are copied together by the new
162745 -        batch file 070405a.bat, created today, see below, ref SDS 0
162746 -        WW3I, into a single file stufqv1 for final processing by batch
162747 -        file 070405.bat.
162748 -
162749 -
162751 -  ..
162752 - Level 4 Virtual Memory Segment for Sub-processing Over 1800 Links
162753 -
162755 -  ..
162756 - Line 3140, ref OF 1 IJ3J, -label level4 in 070405
162757 -
162758 -    -label level4
162759 -    -gosub rfnCs
162760 -    *%lln................................... ref SDS 0 FQ8I
162761 -    g stufq n
162762 -    immed a
162763 -    setcnt 290 1
162764 -    -goto lpcn1............................. ref SDS 0 WM6G
162765 -
162766 -        New code that is called in a 4th memory segment, when a record
162767 -        has more than 1800 lines.  -gosub rfnCs reads in the lines from
162768 -        the original record saved in stufq under -label cites.
162769 -        ref OF 1 HW6I, and based on configuration that calculated the
162770 -        line numbers for the get command. ref OF 1 DX8H
162771 -
162772 -
162774 -  ..
162775 - References Subroutine Get from Stufq Each Pass to Process Links
162776 -
162778 -  ..
162779 - Line 3220, ref OF 1 RT9V, -label rfnCs
162780 -
162781 -    -label rfnCs
162782 -    *%lNt................................... ref SDS 0 VG4J
162783 -    g stufq n n
162784 -    recl 320
162785 -    bot
162786 -    immed /
162787 -    loc_cur 3 1
162788 -    -return
162789 -
162790 -        This routine is loaded at the beginning of the routine to
162791 -        process link, also, citations, per above. ref SDS 0 039B g
162792 -        stufq n n is configured undcer cites, and then called once in
162793 -        level 3, and again in sub-level 4, if necessary when there are
162794 -        more than 1800 lines to process.  After reading in References,
162795 -        recl 320 sets wide enough line length to accomodate converting
162796 -        into HTML up two links on the same line.  bot, immed / and
162797 -        loc_cur 3 1 positions next step to read in and process the
162798 -        links.
162799 -
162800 -
162801 -
162803 -  ..
162804 - 070405.bat  Modify Start Op for Multi-level Processing for Links
162805 -
162806 - This had to be changed to
162807 -
162809 -  ..
162810 - Line 30, ref OF 2 LP4O, in 070405.bat
162811 -
162812 -    if exist stufqv* del stufqv* > stufq9
162813 -
162814 -        Added this command to clear any initial record segments for
162815 -        processing links.  Trying to use "if exist" stufqv* to then
162816 -        delete them, so we avoid a "file not found" report.  Also
162817 -        trying to direct system reporting to stufq9 for same purpose,
162818 -        but seems to only partially work.  We need to affirmatively
162819 -        delete stufqv* because stufqv is the temp file name each pass
162820 -        for the header, Summary and contact lines, per above,
162821 -        ref SDS 0 VG5G, and stufqv1 and stufqv2 are the temporary
162822 -        filenames for content lines with links converted into HTML
162823 -        which are then assembled into a single file stufq for final
162824 -        processing.
162825 -
162827 -     ..
162828 -    %AIR% c:\sd\01\e stufq c:\sd\03\070405 > stufq9
162829 -
162830 -        Normal command unchanged to launch converting SDS records into
162831 -        HTML code, this now runs up to the point of converting links
162832 -        into HTML, and then quits, so that processing links can be
162833 -        accomplished in a fresh memory segment.
162834 -
162835 -
162837 -  ..
162838 - Line 50, ref OF 2 NW6F, in 070405.bat
162839 -
162840 -    if exist stufq8 del stufq8 > stufq9
162841 -
162842 -        Added this command to ensure that temp filename does not exist,
162843 -        so when it is opened the get command can be used to read in
162844 -        lines from stufq for processing. The change to multi-levels
162845 -        processing for links, requires changing the temp file name so
162846 -        that stufq can be used to get Reference lines and content lines
162847 -        for processing in the next step.
162848 -
162849 -
162851 -  ..
162852 - 070405.bat Launch New Memory Segment to Process Links
162853 -
162854 - Line 50, ref OF 2 GV6I, in 070405.bat
162855 -
162856 -    %AIR% c:\sd\01\e stufq8 c:\sd\03\070405 #3 > stufq9
162857 -
162858 -        Added this new step today that processes links separately from
162859 -        everything else, since it is the most memory intensive process.
162860 -        If there are more than 1800 lines another memory segment is
162861 -        called for a sub-level processing.  If a record has more than
162862 -        1800 lines, then a 4th memory segment is opened to process the
162863 -        lines from 1800 to the end of the file, per below. ref SDS 0
162864 -        WW3I
162865 -
162867 -     ..
162868 -    copy stufqv+stufqv1 stufq > stufq9
162869 -
162870 -        Copies of the record back into a single file to complete
162871 -        processing. stufqv is the header, summary and contacts stufqv1
162872 -        is the content with links converted.
162873 -
162874 -
162875 -    %AIR% c:\sd\01\e stufq c:\sd\03\070405 #4 > stufq9
162876 -
162877 -        Third op convert anchors to HTML and complete
162878 -
162879 -
162880 -
162882 -  ..
162883 - 070405a.bat  4th Level Memory Segment Process Links Above 1800 Lines
162884 -
162885 - This is a new batch op called within 070405, per above, ref SDS 0
162886 - 4R7L, when more than 1800 lines are detected in an SDS record.
162888 -  ..
162889 - Line 30, ref OF 3 4V4M, in 070405a.bat
162890 -
162891 -    %AIR% c:\sd\01\e stufq8 c:\sd\03\070405 #5 > stufq9
162892 -
162893 -        Run standard command string for Windows start command using
162894 -        stufq8, which is established to be blank in the primary batch
162895 -        file op in 070405.bat, per above. ref SDS 0 069P  We need a
162896 -        blank file, so that the get command control the parts of the
162897 -        SDS record to process, calling 070405 #5, which is another new
162898 -        part of the code that processes a secondary level for
162899 -        converting links into HTML. ref SDS 0 G28O
162901 -     ..
162902 -    copy stufqv1+stufqv2 > stufq9
162903 -
162904 -        When processing ends, this copy command assembles the two files
162905 -        stufqv1 and stufqv2 into a single file stufqv1, which is then
162906 -        ready for final processing after this 4th level memory segment
162907 -        closes and control passes back to 070405.bat, per above.
162908 -        ref SDS 0 WV4F
162909 -
162910 -
162911 -
162912 -
162913 -
162914 -
162915 -
162916 -
162917 -
162918 -
1630 -