THE WELCH COMPANY
440 Davis Court #1602
San Francisco, CA 94111-2496
415 781 5700


S U M M A R Y


DIARY: July 23, 1999 09:51 PM Friday; Rod Welch

Revise HTML conversion op to avoid running out of memory.

1...Summary/Objective
2...0704050 - Launch Nested DOSE Op for Large Files to Converve Memory


..............
Click here to comment!

CONTACTS 

SUBJECTS
SDS, Editor Convert to HTML
Memory, Multiple Processors, 070405, 0704051
Concept Objectives
Memory Management
Links, Citations
070405 Convert SDS Record to HTML Failing on Large Record

0808 -    ..
0809 - Summary/Objective
0810 -
081001 - Follow up ref SDS 10 0000, ref SDS 9 0000.
081002 -
081003 - Had to revise 070405 that converts SDS records to HTML, added a
081004 - secondary nesting process using dose.
081005 -
081006 -     [On 010604 changed the switch to -label 070405_2 below -label
081007 -     cites. ref SDS 12 0001
081008 -
081009 -
081010 -
0811 -
0812 -
0813 - Problem
0814 -
081401 - The record reviewing Andy Grove's book on 980307, ref SDS 1, will no
081402 - longer convert to HTML, because Medit memory crashes.
081403 - ..
081404 - Line 2000, ref OF 1 7289, -label Distr, out of memory crash
081405 - occurs during this operation.
081406 -
081407 -
081408 -
081409 -  ..
0815 -
0816 -
0817 - Planning
0818 -
081801 - Solution is to try nesting multiple sessions.
081802 -
081803 - On 980405 created level 1 nesting for 070405. ref SDS 2 0001
081804 -
081805 - This has worked well, but I suspect memory is getting chunked up on
081806 - this particular record, such that extending the length of more lines,
081807 - simply does not have any memory areas large enough to accomodate the
081808 - incremental increase.
081809 -
081810 - I checked DOS mem and it indicates there should be about 100K of RAM
081811 - available, but the op crashes anyway.
081812 - ..
081813 - We can split the process into two operations, so the second half
081814 - occurs with a fresh area of memory.
081815 -
081816 -     [On 010604 changed the switch to -label 070405_2 below -label
081817 -     cites. ref SDS 12 0001
081818 -
081819 -
081820 -
081821 -
081822 -
081823 -
0819 -
0820 -
0821 - Progress
0822 -
082201 -  ..
082202 - 0704050 - Launch Nested DOSE Op for Large Files to Converve Memory
082203 -
082204 - Follow up ref SDS 10 9920.
082205 -
082206 - Line 1660, ref OF 1 4248, -label cites this code begins process of
082207 - adding HTML to citations for References.
082208 -
082209 - Line 2000, ref OF 1 7289, -label Distr, out of memory crash occurs
082210 - during this operation.
082211 -
082212 -     The operation adds HTML for all of the links, including bolding
082213 -     each line. Typically there are a lot of these, so maybe 20% of the
082214 -     lines in the file are lengthened by about 100 bytes, using split
082215 -     and join and ins_text.
082216 -
082217 -      ..
082218 -     The file begins with 122K and grows to 183K, about 50%.
082219 -
082220 -     Mem is showing there is 142K still available, so the problem must
082221 -     relate to how the memory is being used.
082222 -
082223 -     Starting with a new block of RAM may solve the problem.
082224 -
082225 -  ..
082226 - Line 2180, ref OF 1 7289, -label diHer
082227 -
082228 - Created a switch based on the size of the file, to launch a secondary
082229 - dose nesting op if number of lines in the file is greater than 1800.
082230 -
082231 -     [On 010604 changed the switch to -label 070405_2 below -label
082232 -     cites. ref SDS 12 0618
082233 -
082234 - We save both 070405 so it has parameters created at the beginning of
082235 - the first pass to save the object file.  The object file, stufq, is
082236 - saved for processing with 070405 again, beginning on line 5, to
082237 - continue processing at -label diHer, where the code finishes on the
082238 - first pass.
082239 - ..
082240 - This method takes a little longer, but not much, even on a
082241 - 300Mhz system, which is where this work is being done today in Concord
082242 - on CPU10.
082243 -
082244 - It seems to solve the problem for now.
082245 -
082246 -     We are probably on the ragged edge of another memory failure with
082247 -     this thing.
082248 -
082249 -   [On 010603 had to fix problem with this code. ref SDS 12 0618
082250 -
082251 -
082252 -
082253 -
082254 -
082255 -
082256 -