THE WELCH COMPANY
440 Davis Court #1602
San Francisco, CA 94111-2496
415 781 5700
rodwelch@pacbell.net


S U M M A R Y


DIARY: November 13, 2003 07:32 AM Thursday; Rod Welch

Called Morris to improve get command for sorting subject indexes.

1...Summary/Objective
2...Problem Opening Large Sorted Subject Index Leads to Segmenting
3...Split Program Divides Large Files into Smaller Files
4...Gary Found a Program to Split Large Files into Smaller Ones
....Split Program Needs Dll File from Microsoft
5...Get Command Enhanced to Read Parts of Large Files Up to 64K Lines
6...Medit New Command for File Last Line Number FLLN
7...Last Line Number in File FLLN New Medit Command Needed
........Compiler Space Solution Divide the File into Smaller Pieces
8...Medit Failing to Process Code Correctly Example Segment SI
9...Medit Small Error in Copyright Notice Caused Program to Fail
......copy_right_msg db " (C) 1989-2003, by Morris Jones. "
......copy_right_msg db " (C) 1989-03, by Morris Jones. "

ACTION ITEMS.................. Click here to comment!

1...Discussed requirements for a new command to get line number...

CONTACTS 
0201 - Intel Corporation                                                                                                                                                  O-00000704 0201
020101 - Mr. Morris E. Jones; Business Unit Manager
020103 - Cable Network Operation                                                                                                                                          O-00000704 0201

SUBJECTS
Medit Get Command Create Alternate with Parameters to Read in a File be
Subject Index Memory Problem Solve with Start Command to Manage Organ
Segment Index into Smaller Indexes Chained Together that Fit SDS Memo
Large Sorted Subject Index Files Segment into Smaller Indexes Chained
Large Index Segment into Smaller Indexes Chained Together that Fit SD
Segment Index into Smaller Indexes that Fit SDS Memory Limitations
Subjects Chain Segmented Indexes Solve Memory Problem with Indexes Se
Segment Subject Index SDS Marketing Sales Points Index Into Smaller F
Segment Chain Subject Indexes Chained for Organic Subject Structure a
Segment Sorted Subject Index into Smaller Indexes Chained Together th
Medit Get Command Create Alternate with Parameters to Read in a File
Get Command Modified to Support Virtual Memory for Sorted Indexes

1514 -
1514 -    ..
1515 - Summary/Objective
1516 -
151601 - Follow up ref SDS 8 0000, ref SDS 7 0000.
151602 -
151603 - Morris improved Medit to enable better memory management handling
151604 - large sorted files for subject index operations. ref SDS 0 03A5  While
151605 - working on this work earlier in the morning, discovered a new problem
151606 - with Medit improvements yesterday. ref SDS 0 PF3H  Morris worked on
151607 - this problem after lunch.  Efforts to use Netmeeting failed, but we
151608 - made progress through discussion on the phone, and using the files
151609 - previously submitted to Morris. ref SDS 0 HR7H  After an hour or so,
151610 - seem to have solved the problem from yesterday by making a very small
151611 - change, which seems out of porportion to the size of the problem it
151612 - caused. ref SDS 0 PRVV  In a letter to Morris on the "get" command
151613 - enhancements, mentioned possibility of solving the compiler problem
151614 - with a system of splitting files.  Gary proposed this for segmenting
151615 - sorted subject indexes into smaller files, and he submitted a program
151616 - to support his task. ref SDS 0 PQUW  Morris did not have time to
151617 - review the compiler issue today. ref SDS 0 1B5H
151619 -  ..
151620 - Submitted an email to Gary thanking him for input on the split file
151621 - objective, and reporting current status. ref SDS 0 PQUW  Sent a
151622 - letter to Morris thanking him for work on Medit.
151623 -
151624 -     [On 040404 Gary commends Morris' collaboration and support for
151625 -     SDS. ref SDS 13 FQ5J
151626 -
151627 -
151628 -
151629 -
151630 -
151631 -
151633 -  ..
1517 -
1518 -
1519 - Progress
1520 -
152001 - Problem Opening Large Sorted Subject Index Leads to Segmenting
152002 -
152003 - Yesterday, the account in the SDS Subject Index for Gary
152004 - supporting SDS development would not sort.  Continued experimentation
152005 - caused the screen to become completely confused with extraneous
152006 - control characters and odd coloration, indicating a low level memory
152007 - issue.  The file at issue is about 215K, which extends limits of Medit
152008 - memory, which itself is over 200K.
152010 -  ..
152011 - Decided to segment the sorted lists and chain them together similar to
152012 - the procedure developed on 030810 for the standard index files.
152013 - ref SDS 5 0001
152014 -
152015 -       [...below, the actual cause of this problem may be due to an
152016 -       error in Medit code that occurred yesterday, ref SDS 0 PF3H,
152017 -       that was fixed later today, per below. ref SDS 0 PRVV
152019 -  ..
152020 - Did some preliminary work revising the code slightly to consolidate
152021 - two (2) dose calls into one. ref OF 2 OI3I and ref OF 2 1I5L
152023 -  ..
152024 - To manage segmented sorted index files, we need to read in specified
152025 - portions of a file with the get command.
152026 -
152027 -       [On 031114 changed the design scheme to read sorted index files
152028 -       in 17 line chunks and developed new functionality to navigat
152029 -       through large files by scrolling with page up and down.
152030 -       ref SDS 9 0001
152032 -  ..
152033 - Talked to Gary.
152035 -  ..
152036 - He recalled that programs are available that split files into smaller
152037 - parts, and related that this is a subject on the Internet.
152039 -  ..
152040 - Did some research and found information about software programs that
152041 - divide files.
152043 -  ..
152044 - Sent a letter to one vendor.
152046 -  ..
152047 - Got a letter back saying his program does not enable specifying line
152048 - numbers.
152050 -  ..
152051 - Submitted ref DIT 1 0001 to Morris saying...
152052 -
152053 -    1.  I need a "get" command that can specify the lines to read into
152054 -        a file, something like...
152055 -
152056 -              g fred 800 400
152057 -
152058 -        ...where fred is the filename and the get command begins
152059 -        reading at line 800 to the next 400 lines or to line 1200.
152060 -
152061 -           [On 040404 Gary commends Morris' collaboration and support
152062 -           for SDS. ref SDS 13 FQ5J
152064 -         ..
152065 -    2.  If fred only has 1000 lines then....
152066 -
152067 -              g fred 800 400
152068 -
152069 -        ...would get the last 200 lines, i.e., it does not crash when
152070 -        param is not met.  Similarly, if fred only has 700 lines,
152071 -        then...
152072 -
152073 -              g fred 800 400
152074 -
152075 -        ...simply reads nothing into the current file.
152077 -         ..
152078 -    3.  This capability enables sorting multiple files into a big list
152079 -        and then chaining them together to create a form of "virtual"
152080 -        memory that does not exceed Medit's limits.  Have developed
152081 -        this for the Subject Index and it works well.  The alternate
152082 -        get command would extend this support.
152083 -
152084 - ...below, also ask in the same letter about the compiler problem.
152085 - ref SDS 0 JU6W
152086 -
152087 -
152088 -
152090 -  ..
1521 -
1522 -
1523 - 0748
1524 -
152401 - Called Morris.
152402 -
152403 - He had not yet seen the letter submitted earlier this morning.  We
152404 - reviewed the objectives, ref SDS 0 MP8F,
152406 -  ..
152407 - Morris asked about scope and use cases for the revised command.  We
152408 - discussed the parameters in the letter to Morris, per above.
152409 - ref SDS 0 WR52  He will review the letter.
152410 -
152411 -        [On 040404 Gary commends Morris' collaboration and support for
152412 -        SDS. ref SDS 13 FQ5J
152414 -  ..
152415 - I also mentioned we may have a problem with the work done yesterday,
152416 - per below. ref SDS 0 PF3H
152418 -  ..
152419 - Morris checked his computer and advised he has a long conference call
152420 - scheduled for this morning.  He wants to work on Medit after lunch.
152421 -
152422 -     [...below, called Morris before the conference call with Jerry and
152423 -     Gary, ref SDS 0 J94L,
152424 -
152425 -
152426 -
152427 -
152429 -  ..
1525 -
1526 -
1527 - 0818
1528 -
152801 - Split Program Divides Large Files into Smaller Files
152802 - Gary Found a Program to Split Large Files into Smaller Ones
152803 -
152804 - Received a letter from Gary saying to try a program he found on his
152805 - computer....
152806 -
152807 -             split.exe
152809 -  ..
152810 - Called and talked to Gary.
152812 -  ..
152813 - He clarified how to issue the command....
152814 -
152815 -    C>split -l 300 fred
152816 -
152817 - ...should split fred into files with no more than 300 lines each.
152818 -
152819 -
152821 -     ..
152822 -    Split Program Needs Dll File from Microsoft
152823 -
152824 -
152825 - Tried this and got an error message saying...
152826 -
152827 -          win32gnu.dll
152828 -
152829 - ...cannot be found on the system path.
152831 -  ..
152832 - Evidently split is using Windows utilities.
152834 -  ..
152835 - Could not find win32gnu.dll anywhere on c13.
152836 -
152837 -     [...below, letter to Morris asks about using a program to split
152838 -     large files into smaller pieces to solve the compiler environment
152839 -     space problem. ref SDS 0 JU6W
152840 -
152841 -
152842 -
152843 -
152844 -
1529 -

SUBJECTS
Get Command Improved with New Parameters to Specify Line Number to Be
Medit Get Command Create Alternate with Parameters to Read in a File
Morris Developes Revised Medit Get Command Create Alternate with Para
Medit Get Command Create Alternate with Parameters to Read in a File
Get Command Create Alternate with Parameters to Read in a File beginn

2107 -
2108 - 1536
210901 -  ..
210902 - Get Command Enhanced to Read Parts of Large Files Up to 64K Lines
210903 -
210904 - After fixing the problem with Medit caused by changes yesterday, per
210905 - below, ref SDS 0 PF3H, Morris began work to enhance the get command,
210906 - per the letter this morning, shown above. ref SDS 0 WR52
210908 -  ..
210909 - Morris asked how many lines a file could have that will be processed,
210910 - and indicated that files with more than 32K lines require additional
210911 - code.
210913 -  ..
210914 - Upon review, decided that the current application does not contemplate
210915 - files with more than 32K, however, if greater flexibility can be
210916 - developed to support files with 64K lines without a lot of extra time
210917 - and effort, lets do that.  As a result, Morris gave the new command
210918 - capacity to process a file with 64K lines.
210919 -
210920 -        [On 040404 Gary commends Morris' collaboration and support for
210921 -        SDS. ref SDS 13 FQ5J
210923 -  ..
210924 - Morris asked about command syntax....
210926 -  ..
210927 - Common syntax is listed in the letter earlier today, per above.
210928 - ref SDS 0 WR52
210930 -  ..
210931 - Morris gave examples...
210932 -
210933 -      g fred 0 10
210934 -
210935 - ...will not get anything.
210936 -
210937 -      g fred 100 0
210938 -
210939 - ...will not get anything.
210940 -
210941 -
210942 -
210944 -  ..
2110 -
2111 -
2112 - 1621
2113 -
211301 - Morris completed work, and submitted via email a revised file....
211302 -
211303 -                     c: 00 10 getf.asm
211304 -
211306 -  ..
211307 - Installed this and ran a test.  Seems to work, per spec submitted in
211308 - the letter to Morris, shown above. ref SDS 0 WR52
211310 -  ..
211311 - Created a record in the Medit command file, ref OF 7 RN4K, with
211312 - examples of syntax, per above. ref SDS 0 WR52
211313 -
211314 -     [On 031114 applied to SI code in 03501 to enhance managing sorted
211315 -     subject index files that avoid exceeding Medit memory limits.
211316 -     ref SDS 9 WI7I
211318 -      ..
211319 -     [On 031114 inital testing shows new get command features helpful
211320 -     to reduce memory management problem; new problem using get command
211321 -     may show a path to fixing another part of Medit that causes the
211322 -     program to fail after continued use. ref SDS 10 0001
211324 -      ..
211325 -     [On 040404 Gary commends Morris' collaboration and support for
211326 -     SDS. ref SDS 13 FQ5J
211328 -      ..
211329 -     [On 050303 applied new get command for binary search in Subject
211330 -     Index. ref SDS 14 5F7M
211331 -
211332 -
211333 -
211334 -
211335 -
211336 -
2114 -

SUBJECTS
File Last Line Number FLLN N Filename New Command Needed to Facilitat
Last Line Number File FLLN N Filename New Command Needed to Facilitat

2504 -
250501 -  ..
250502 - Medit New Command for File Last Line Number FLLN
250503 - Last Line Number in File FLLN New Medit Command Needed
250504 -
250505 -
250506 - Discussed requirements for a new command to get line number...
250507 -
250508 -
250509 -                     flln n [path filename]
250510 -
250511 - ...to read a file on the disk and put the last line number in a
250512 - counter (n).  This facilitates navigating in knowledge space by
250513 - jumping to the bottom of the file with a single command when
250514 - scrolling through a file to find information and reading only chunks
250515 - of memory in 17 line increments.
250517 -  ..
250518 - Morris feels he can do this.
250519 -
250520 -    [On 031114 developed solution to immediate requirement to support
250521 -    sorting subject index by using output from Opt Tech sort program to
250522 -    get the line number. ref SDS 9 WI7I
250523 -
250524 -    [On 031116 letter to Morris lists flln command as pending matter.
250525 -    ref SDS 11 YY8R
250527 -     ..
250528 -    [On 040404 Gary commends Morris' collaboration and support for SDS.
250529 -    ref SDS 13 FQ5J
250530 -
250531 -
250532 -
250533 -
250534 -
250535 -
2506 -

SUBJECTS
Compiler Space Problem Divide Split Files

2603 -
260401 -         ..
260402 -        Compiler Space Solution Divide the File into Smaller Pieces
260403 -
260405 -  ..
260406 - Continuing the letter to Morris and following up on Gary's letter this
260407 - morning, per above, ref SDS 0 PQUW, submitting a program that can
260408 - split large files...
260410 -         ..
260411 -    4.  Before hitting on the idea to use the get command, did some
260412 -        research to find a utility that divides files. ref DIT 1 2X8G
260413 -        This yielded the following...
260414 -
260415 -            http://www.codefarms.com/docs/dol/ch09sngu.htm#9_3
260417 -         ..
260418 -        ...that says in part...
260419 -
260420 -           "Some compilers, especially under DOS, run out of space when
260421 -           applied to a large file which contains many functions.  The
260422 -           solution to this problem is to split the file into several
260423 -           files, each containing fewer functions and fewer lines of
260424 -           code"
260426 -         ..
260427 -    5.  Could this approach help solve the compiler problem we are
260428 -        having?  Is it possible to compile the program in smaller parts
260429 -        and then assemble the parts?
260430 -
260432 -  ..
260433 - We did not have time today for Morris to comment on this.  Not sure
260434 - if he has reviewed the source.  Will follow up separately.
260435 -
260436 -
260437 -
260438 -
260439 -
260440 -
260441 -
260442 -
260443 -
2605 -

SUBJECTS
Medit Failing to Perform Subject Index Segment Function Running New C
Medit Failed Perform Subject Index Segment Function Running New Code

2804 -
280501 -  ..
280502 - Medit Failing to Process Code Correctly Example Segment SI
280503 -
280504 - Follow up ref SDS 8 4N7L.
280505 -
280506 - This morning while reporting on research and development of a new
280507 - system to manage context, per above, ref SDS 0 MP8F, noticed that the
280508 - subject index for Morris to support SDS development was nearing memory
280509 - limits, so ran the function developed on 030810 to segment and chain
280510 - the index. ref SDS 5 0001  The code failed to function.
280512 -  ..
280513 - Ran this a few more times and confirmed the code failed.
280514 -
280515 -     [On 040102 similar kind of error in Medit, but harder to find and
280516 -     correct. ref SDS 12 599M
280518 -  ..
280519 - Tried debug and the code seemed to suddenly stop without any evident
280520 - error in SDS code.  Ran the same op on c11 and it ran correctly.
280521 - Thought possibly the change made yesterday to begin improving the
280522 - feature to sort indexes might have caused the problem, so copied the
280523 - macro file 03501, which runs the SI, from c11 to c13.  This did not
280524 - solve the problem.
280526 -  ..
280527 - Moved all SDS code on c13 to another directory and copied the entire
280528 - body of code from c11 to c13.  This solved the problem.
280530 -  ..
280531 - Compiled e.exe again, using the code in the Medit directory, c: 00
280532 - 10, and ran the test again, and the thing crashed.
280534 -  ..
280535 - Moved files from c13 c: 00 10 to another directory, and copied the
280536 - files from c11 c: 00 10 which were preserved from 031109, the day
280537 - before Morris worked on the code.  Compiled Medit into e.exe and ran
280538 - the test again.  There was no problem.
280540 -  ..
280541 - This record suggests something in the code Morris developed on 031110
280542 - is defective.
280543 -
280544 -
280546 -  ..
2806 -
2807 - 1241
2808 -
280801 - Called Morris.
280802 -
280803 - He advised that long conference call scheduled for this morning did
280804 - not happen, so he has looked at the issues on Medit, per our call this
280805 - morning and the letter to Morris, explained above. ref SDS 0 QO8C
280807 -  ..
280808 - Advised there is a conference call at 1300 with Gary and Jerry on
280809 - applying SDS for Com Metrics, and Jerry's progress learning SDS.  This
280810 - can take 30 minutes or so.  Asked if he would like to join the
280811 - discussion.
280813 -  ..
280814 - Morris wants to focus on correcting the Medit problem and develop the
280815 - new enhancement.
280817 -  ..
280818 - I will call him after the conference call.
280819 -
280820 -
280821 -
280822 -
280824 -  ..
2809 -
2810 -
2811 - 1402
2812 -
281201 - Called Morris.
281202 -
281203 - We worked for half an hour or so, but could not get Netmeeting to run.
281204 - He feels the problem may be that we are both using a router that
281205 - firewall protection.  Evidently he can use Netmeeting at Intel where
281206 - there is a lot of firewall protection, but for some reason Netmeeting
281207 - cannot navigate through our commercial routers.
281209 -  ..
281210 - We considered using Microsoft Instant Messanger system.  Turned out
281211 - this is not installed on my computer.  We tried to install from the
281212 - original code, but it is not on the CD as an accessory.
281214 -  ..
281215 - Abandoned this effort.
281217 -  ..
281218 - We looked at the Medit program code in c: 00 10.
281220 -  ..
281221 - Started doing some compares to see what Morris changed yesterday.
281222 -
281224 -  ..
2813 -
2814 -
2815 - 1516
2816 -
281601 - Medit Small Error in Copyright Notice Caused Program to Fail
281602 -
281603 - Morris suggested looking in medit.asm at a change he made yesterday to
281604 - the copyright notice at line 410. ref OF 5 1N6G
281606 -  ..
281607 - This line said....
281608 -
281609 -      copy_right_msg  db " (C) 1989-2003, by Morris Jones. "
281610 -                                    ----
281611 - Morris suggested changing it to...
281612 -
281613 -      copy_right_msg  db " (C) 1989-03, by Morris Jones. "
281614 -                                    --
281615 -
281616 - ...thus, we changed "2003" to "03" removing two bytes "20"
281618 -  ..
281619 - Compiled the program and it ran successfully.
281621 -  ..
281622 - This problem consumed about 5 - 6 hours today chasing down 2
281623 - characters.
281625 -  ..
281626 - Morris expects that the extra two (2) characters for the copyright
281627 - string he added wound up exceeding the memory allocated for that field
281628 - and so this had a ripple effect that caused unintended consequences.
281629 - It is interesting the problem does not show up immediately when the
281630 - program is opened, but only impacts things in a subtle manner,
281631 - essentially masking the problem.
281632 -
281633 -     [On 040102 similar kind of error in Medit, but harder to find and
281634 -     correct. ref SDS 12 599M
281636 -  ..
281637 - Upon review, while segmenting a subject index is not related in SDS
281638 - code to sorting an index, the problem sorting the index for Gary,
281639 - reported above, ref SDS 0 QN4J, that led to work on developing the
281640 - "get" command, also, reported above, ref SDS 0 03A5, could actually be
281641 - caused by the same problem that prevented segmenting the index for
281642 - Morris, per above. ref SDS 0 PF3H
281644 -  ..
281645 - After fixing the Medit problem, we worked on the get command
281646 - enhancement, per above. ref SDS 0 AT6E
281647 -
281648 -
281649 -
281650 -
281651 -
281652 -
281653 -
281654 -
281655 -
281656 -
281657 -
2817 -
Distribution. . . . See "CONTACTS"