Please follow the instructions in bold-italics and answer the questions that follow.

WWW.OREILLY.COM

Log on to the O'Reilly web site (www.oreilly.com) and look over the information provided on the home page.

1. What is your general impression of the O'Reilly home page? Please check one.
   1. Too much information

      Select: yes

   2. Too little information

      Select: no

   3. Just the right amount of information

      Select: no

2. What information, if any, do you find particularly helpful on this home page? Please describe.

   Answer:

   Hot off the press -- is helpful to show newly released books.

   Frankly Speaking -- is a good concept the way it is described to provide opinion to distinguish useful trends from passing fads. However, reading the referenced web page was disappointing because the writing takes too long to make its points. This is a matter of taste. No doubt many people may be attracted to the site by the Editor's focus on humor.

   Resource Centers -- in the left margin, seems to provide some catagories of books available through Oreilly.

3. What information, if any, do you feel is missing from this home page?

   Answer:

   The print is too small.

   Took quite awhile to find "Special Interest" requested under para 7 below.

   Home page has too much stuff, "can't see forest for the trees."

   There is no conceptual explanation of the kind of books or service offered by O'Reilly. It is not clear whether Oreilly provides merely books or whether software tools and programs can be purchased at this site as well.

   This may be less of a problem if the site is typically used only by people familiar with O'Reilly products and services, e.g., computer programmers.

   If Oreilly provides "online" books, as seems to be the case, based on para 8, then it would be helpful to have this set out clearly, including, perhaps in a link, terms and conditions for downloading or obtaining CDs.

   Instruction

   Imagine you are preparing to take an MCSE exam on Exchange Server. Look for books on this topic on the O'Reilly web site and evaluate 2-3 books to see if they will meet your needs.

4. How would you rate your experience looking for books on this web site? Please circle one.
   1. Very Difficult

      Select: no

   2. Somewhat Difficult

      Select: no

   3. Neither Easy Nor Difficult

      Select: no

   4. Somewhat Easy

      Select: no

   5. Very Easy

      Select: yes

   Please explain your response. What, in particular, made your search easy or difficult?

   Answer:

   The Oreilly Search program found about 20 titles using "MCSE" and the first two seem to provide support for an exam.

   Initially, I entered the search string incorrectly as "MSCE" and got no hits. I looked for "Exchange Server" and got no hits. Then examined the Product Index in the left margin, and found in the alphabetical list several selections for MCSE. Went back to the search window and entered "MCSE" and this time got the list of 20 hits.

5. What information did you use to determine whether or not a book would meet your needs? Please check all that apply.
   1. "About the Author"

      Select: no

   2. Author name

      Select: no

   3. Book title

      Select: yes

   4. Edition

      Select: yes

   5. Full description about the book

      Select: yes

   6. Index

      Select: no

   7. ISBN

      Select: no

   8. Number of pages

      Select: no

   9. Price

      Select: yes

   10. Publication Date

       Select: no

   11. Reader reviews

       Select: no

   12. Related O'Reilly titles

       Select: no

   13. Short description about the book

       Select: yes

   14. Table of contents

       Select: yes

   15. Other:

       Describe:

       
       
       

6. What other information, if any, would have been helpful for evaluating books on this topic?

   Answer:

   It might be useful to know how this source relates to other popular titles on the same subject, since it is intended for guidance on taking an examination. Is it the most popular text, most comprehensive, does it offer advantages over the most popular source?

   
   Instruction

   Click on the "O'Reilly" name in the left-hand column to return to the O'Reilly home page.

   In the left-hand column of the home page, scroll down to the "Special Interest" Section and click on the Ask Tim button.

   Evaluate the information provided in the Ask Tim section.

7. How helpful do you consider the information provided in the "Ask Tim" section of the O'Reilly web site? Please circle one.
   1. Not Helpful At All

      Select:

   2. Not Very Helpful

      Select:

   3. Somewhat Helpful

      Select: yes

   4. Very Helpful

      Select:

   Please explain your response:

   Answer:

   There was one question to Tim about a potentially useful idea of putting chapters or whole books online prior to publication so subscribers can provide feedback.

   Tim indicates accomplishing this goal requires coordination between the editor of the Oreilly web site and manager of online books.

   Is there a feedback loop to the inquirer from the website editor and the manager of online books?

   Does Tim or somebody initate a survey to ascertain how valuable the proposal might be to Oreilly, so that cost/benefit is assessed?

   Such feedback can keep inquires coming to Ask Tim, so that it is more than just a perfunctory letter to the editor section.

   The headings in the right margin are not helpful to me personally, but seem comprehensive. I checked a few titles and found the dialog direct and to the point.

   Examined "Win95 in a Nutshell Questions." This led to a reference which I tried to grap with cut and paste to load into this document for discussion. Could not do this. The browser is grabbing the entire several paragraphs. In any case, opening the session showed another reference that was of interest, but it was not accessible.

   Instruction

   Click on the Books Online button in the left-hand column of the web site and read through the information provided about O'Reilly's online book offerings.

8. How interested would you be in using an online book like those offered by O'Reilly? Please circle one.
   1. Not Interested At All

      Select:

   2. Not Very Interested

      Select:

   3. Somewhat Interested

      Select:

   4. Very Interested

      Select: yes

   Please explain your response:

   Answer:

   Online books are a key part of Communication Metrics goals for a "paperless office." They support positioning information in knowledge space so that connections can deliver intelligence via Internet, as explained in the record on Jan 25, 1999. Online books or on a CD that can be downloaded and integrated into a data base on all subjects provide a significant advance in learning and utility relative to traditional printed media.

   Online manuals like Oreilly's HTML The Definitive Guide, may avoid problems of misreading command strings because the printed text is blurred, or inconclusive, as shown in the record on Jan 25, 1999.

   
   Instruction

   Click through and explore some of the Resource Centers listed on the left-hand column of the O'Reilly web site to get a feel for the type of information that is provided.

9. How helpful do you consider the information provided in these Resource Centers? Please circle one.
   1. Not Interested At All

      Select:

   2. Not Very Interested

      Select:

   3. Somewhat Interested

      Select:

   4. Very Interested

      Select: Yes

10. What types of information did you find especially helpful?

    Answer:

    Most of this information seems helpful.

    Open Source and in particular Linux provides useful information on possible emerging trends and options to DOS/Windows. I did not take time to chase it down, but expect there is an explanation of advantages and impacts that result from moving to an alternative.

    PalmPilot provides a quick way to keep up with progress in this field.

    Annoyances provides sources to enhance performance of w98, and other MS products. I bookmarked this section for further review.

    In a Nutshell seems to provide quick guidance on a variety of programming systems, although there does seem to be a w98 In a Nutshell reference source as yet.

    Web-Based Training is valuable to extend rudimentary web page development.

    Specials offers reduced pricing that can save money when shopping for related reference books.

    Writing for Us explains procedures for authors to publish through O'Reilly, which may provide support for wider publication of Welch papers. I took this occassion to submit a proposal.

11. What types of information did you find less helpful?

    Answer:

    Ask Tim seems the least helpful to me, although this may only reflect prior frustrations having submitted letters to the editor and not getting any feedback owing to the enormous burden on editors when faced with a deluge of submissions.

12. Please note any other comments or impressions you may have about the O'Reilly web site here:

    Answer:

    Seems like the site is slow.

    It would be helpful if it were possible to switch to a larger font, so it is easier to read. Of course the layout and design of the site would be impacted, but it might be a good trade-off, if it could be done without significantly increased cost.

1. What is your general impression of the Microsoft Press home page? Please check one.
   1. Too much information

      Select:

   2. Too little information

      Select:

   3. Just the right amount of information

      Select: Yes

2. What information, if any, do you find particularly helpful on this home page? Please describe.

   Answer:

   Print is a little bigger and the page less crowded than Oreilly.

   There is a link to an explanation of the site that catches the eye immediately so that first time visitors can get an orientation.

3. What information, if any, do you feel is missing from this home page?

   Answer:

   If Microsoft Press offers online books, this is not evident from the home page.

   The print in the left margin is very hard to read. For example, question 7 asks for review of the Training section, and this is hard to find.

   
   Instruction

   Imagine you are preparing to take an MCSE exam on Exchange Server. Look for books on this topic on the Microsoft Press web site and evaluate 2-3 books to see if they will meet your needs.

4. How would you rate your experience looking for books on this web site? Please circle one.
   1. Very Difficult

      Select: no

   2. Somewhat Difficult

      Select: no

   3. Neither Easy Nor Difficult

      Select: no

   4. Somewhat Easy

      Select: no

   5. Very Easy

      Select: yes

   Please explain your response. What, in particular, made your search easy or difficult?

   I noticed in the left margin a listing for "MCP Exam," and this seemed like it might be another way for identifying references to support an MSCE Exam. Opening this page seemed to show it provides information to prepare for the MSCE exam.

   Did a search for MSCS Exam and got a list of sources that cited the MCP Exam, so this seems like it satisfies the objective. Some of these sources were a "Chat" environment, but several books seemed to be listed as well.

5. What information did you use to determine whether or not a book would meet your needs? Please check all that apply.
   1. "About the Author"

      Select:

   2. Author name

      Select:

   3. Book title

      Select: yes

   4. Edition

      Select:

   5. Full description about the book

      Select: yes

   6. Index

      Select:

   7. ISBN

      Select:

   8. Number of pages

      Select:

   9. Price

      Select:

   10. Publication Date

       Select:

   11. Reader reviews

       Select:

   12. Related O'Reilly titles

       Select:

   13. Short description about the book

       Select: yes

   14. Table of contents

       Select: yes

   15. Other:

       Describe:

       
       
       

6. What other information, if any, would have been helpful for evaluating books on this topic?

   Answer:

   See the answer for Oreilly.

   Discovered that Training in the left margin pertained to MSCE Exam training, when doing question 7, below. Consideration might be given to making this easier to recognize.

   
   

   Instruction

   Explore the seven sections of the web site under the Training heading (located in the left-hand column of the web site).

7. Circle the number that indicates how helpful you consider each type of Training information, where 1 = Not helpful at all; 5 = Very Helpful
   1. Success Stories

      Rating: 3

   2. Getting Started

      Rating: 5

   3. Staying Ahead

      Rating: 3

   4. Ask the Experts

      Rating: 4

   5. MCP Exam Info

      Rating: 5

   6. MCP Scholarship

      Rating: 3

   7. MCP Connection

      Rating: 4

8. Please note any other comments or impressions you may have about the Microsoft Press web site here:

   Answer:

   It is not evident that this site offers online books.

   The site is narrowly focused on Micrsoft, but since Microsoft dominates the PC environment, it necessarily is useful.

   The site seems direct in answering "how to" questions, as in in how to do this, that or the other, which is useful given Microsoft's market dominance.

   There is no evident reference material on how to apply Microsoft products and technology usefully, for example, to support thinking, remembering and communicating, as in a generic intelligence role for business explained in the POIMS paper.

   This omission reflects an assumption that Microsoft products are useful, perhaps rightly so based on market share as a metric of usefulness. The orientation of the site managers is on how to use Microsoft products, rather than on the usefulness of the products and how to formulate a conceptual framework that enables development of more useful software, as reviewed in the record on Jan 25, 1999.

1. What is your general impression of the Macmillan home page? Please check one.
   1. Too much information

      Rating: yes, barely

   2. Too little information

      Rating:

   3. Just the right amount of information

      Rating:

2. What information, if any, do you find particularly helpful on this home page? Please describe.

   Answer:

   The listing of prominent books is clear, and the print on these is large enough to handle without undue effort.

   Current Location - Computer Reference - this section provides an explanation of what the site is about and its contents.

   Left Margin Contents - easy to read with large font, and has links with descriptions that seem easy to grasp and identify as useful.

3. What information, if any, do you feel is missing from this home page?

   Answer:

   It would be helpful to me if there was an express statement that the site has available online books. This is redundant once people are familiar with site, but initially this is very useful information to me.

   The flashing message is bothersome.

   The logos at the top of the home page are too crowded, causing initial confusion.

   Instruction

   Imagine you are preparing to take an MCSE exam on Exchange Server. Look for books on this topic on the Macmillan web site and evaluate 2-3 books to see if they will meet your needs.

4. How would you rate your experience looking for books on this web site? Please circle one.
   1. Very Difficult

      Select:

   2. Somewhat Difficult

      Select:

   3. Neither Easy Nor Difficult

      Select:

   4. Somewhat Easy

      Select:

   5. Very Easy

      Select:

   Please explain your response. What, in particular, made your search easy or difficult?

   Answer:

   Initially, tried to use Distance Learning. This was not successful.

   Tried Help and found under Special Offers a listing Click here to take the fast track to becoming a Microsoft Certified Systems Engineer," which seems likely provides support for the MSCE exam. Opened the site and it appears to have a lot of sources that would be helpful in preparing for the MSCE exam.

5. What information did you use to determine whether or not a book would meet your needs? Please check all that apply.
   1. Author name

      Select:

   2. Availability (e.g., "in stock")

      Select:

   3. Book title

      Select: yes

   4. Description of the book

      Select: yes

   5. ISBN

      Select:

   6. Number of pages

      Select:

   7. Price

      Select: yes

   8. Publication date

      Select:

   9. Publisher

      Select:

   10. Other:

       Describe:

       I did not have enough time to review details of the selections in this listing.

6. What other information, if any, would have been helpful for evaluating books on this topic?

   Answer:

   See the answer for Oreilly.

   Since MSCE seems to be a major issue in this industry, it would be helpful to have a specific catagory for this training, as on the Microsoft site

   
   

   Instruction

   Click on the "Personal Bookshelf" button located near the top of the Macmillan web site.

   Read through the information about Macmillan's personal bookshelf feature.

7. How interested would you be in using Macmillan's personal bookshelf? Please circle one.
   1. Not Interested At All

      Select:

   2. Not Very Interested

      Select:

   3. Somewhat Interested

      Select:

   4. Very Interested

      Select: yes

   Please explain your response:

   Answer:

   Please see the answer for Oreilly under question 8.
   

   Instruction

   Click on the Resource Centers button located near the top of the Macmillan web site.

   Explore several of the Resource Centers to get a feel for the type of information that is provided.

8. How helpful do you consider the information provided in these Resource Centers? Please circle one.
   1. Not Helpful At All

      Select:

   2. Not Very Helpful

      Select:

   3. Somewhat Helpful

      Select: yes

   4. Very Helpful

      Select:

9. What types of information did you find especially helpful?

   Answer:

   I did not find any particular resource directly helpful to me, although with more time probably there would be quite a bit that could contribute in various ways.

10. What types of information did you find less helpful?

    Answer:

    It was suprising to find that the link to DOS reference material pertained to DOS 6.22 and there was nothing for DOS under W95 or W98.

11. Please note any other comments or impressions you may have about the Macmillan web site here:

    Answer:

    I like the ease of comprehension of the web site, primarily due to the larger size fonts and the lack of congestion of the Home page.

    I would like to see a link to more subjects, including books online for other subjects rather than solely for computer stuff, e.g., management texts, history, science, etc.

1. What is your general impression of the Computer Literacy home page? Please check one.
   1. Too much information

      Select:

   2. Too little information

      Select:

   3. Just the right amount of information

      Select: yes

2. What information, if any, do you find particularly helpful on this home page? Please describe.

   Answer:

   Welcome to Computer Literacy - this section provides an immediate explanation of what the site is about and its contents. I like this a lot.

   Color and overall design with good contrast between letters and background -- it is not overwhelming to span of attention.

   The size of the letters is about right, so that titles stand out, but the text is easy to read.

   The headings are fairly common and easy to grasp.

3. What information, if any, do you feel is missing from this home page?

   Answer:

   See suggestion for an express statement re online books under question 3 for Macmillian.

   
   Instruction

   Imagine you are preparing to take an MCSE exam on Exchange Server. Look for books on this topic on the Computer Literacy web site and evaluate 2-3 books to see if they will meet your needs.

4. How would you rate your experience looking for books on this web site? Please circle one.
   1. Very Difficult

      Select:

   2. Somewhat Difficult

      Select:

   3. Neither Easy Nor Difficult

      Select:

   4. Somewhat Easy

      Select:

   5. Very Easy

      Select: yes

   Please explain your response. What, in particular, made your search easy or difficult?

   Answer:

   Noticed under the Subjects heading there is a listing for MCSE.

5. What information did you use to determine whether or not a book would meet your needs? Please check all that apply.
   1. Author info

      Select:

   2. Author name

      Select:

   3. Book title

      Select: yes

   4. Books by the same author

      Select:

   5. Description about the book (Summary)

      Select: yes

   6. ISBN

      Select:

   7. Number of pages

      Select:

   8. Price

      Select:

   9. Publication date

      Select:

   10. Publisher

       Select:

   11. Reader reviews

       Select:

   12. Table of contents

       Select:

   13. Other:

       Describe:

       There is a publisher's recommendation for one of the books, indicating it may be an industry standard.

6. What other information, if any, would have been helpful for evaluating books on this topic?
   See the answer for Oreilly.
   

   Instruction

   Click on the "Home" button near the top of the Computer Literacy web page.

   Enter the word "Exchange" in the "Quick Search" field and click on the "Quick Search" button.

7. To the right of your screen is an "Icon Legend". What do you think of this sort of information? How helpful do you consider having symbols like these presented along with the search results?
   This seems moderately useful, as noted in the answer to question 5.13.

8. Near the top of your screen, you will see a Books and a Training tab.

   What do you think of this classification system? Do you prefer to have products separated into Books and Training or do you prefer to have a single list showing both types of products?

   Answer

   It could be useful, if the content of each is clearly enough distinguished.

9. Please note any other comments or impressions you may have about the Computer Literacy web site here:
   Please see the answer to question 8 for Microsoft.
   

BOOK INFORMATION

Now that you have evaluated computer books on several web sites, we'd like your feedback on the different types of book information that can be provided.

1. After each item, please circle the number that indicates how helpful you consider that type of book information when evaluating whether or not a book will meet your needs.

   Rate 1 - 5, where 1 = Not helpful at all; 5 = Very Helpful

   Book Information (Examples)

   1. Audience type (For IT Professionals, For Developers, For General Computer Users)

      Rating: 5

   2. Author biography

      Rating: 3

   3. Author name

      Rating: 4

   4. Book availability (In Stock, Out of Stock, Out of Print)

      Rating: 3

   5. Book format (Paperback, Hardback)

      Rating: 2

   6. Description about the book

      Rating: 4

   7. Description of the disk/CD's contents

      Rating: 4

   8. Disk/CD included

      Rating: 5

   9. Index

      Rating: 4

   10. List of books by the same author

       Rating: 2

   11. List of books on the same or related topics

       Rating: 2

   12. Number of pages

       Rating: 3

   13. Price

       Rating: 3

   14. Publication date

       Rating: 4

   15. Reader reviews

       Rating: 2

   16. Sample chapter or excerpt from the book

       Rating: 4

   17. Shipping time (Ships in 24 hours, Ships in 2-3 days, etc.)

       Rating: 3

   18. Table of contents

       Rating: 4

   19. Transcript from an interview with the author about the book

       Rating: 3

   20. User level (Beginner, Intermediate, Advanced)

       Rating: 3

2. Besides the book title, what types of book information would be the most helpful for making a decision whether or not a book will meet your needs?

   Please insert the letter of each item you select from the list above. You may list up to five items.

   1. b
   2. f
   3. h
   4. m
   5. p

3. What additional information would be the most helpful for deciding whether or not a book will meet your needs? Please insert the letter of each item you select from the list above. You may list up to five items, excluding those you listed in Question #2.
   1. o
   2. d
   3. e
   4. l
   5. n

COMPUTER BOOK WEB SITE FEATURES & SERVICES

1. Please review each of the following computer book web site features & services and circle the number that indicates your likelihood of using each one.

   Rate 1 - 5, where 1 = definitely would not use; 5 = definitely would use

   1. Advance ordering: the ability to reserve a copy of a book in advance of its publication.

      Rating: 1

   2. Bestseller list: a list of the top selling books during the past month.

      Rating: 4

   3. Certification and training information: information on how to become a certified professional (e.g., MCSE, CNE, etc.) and the resources you can use to prepare for certification exams.

      Rating: 1

   4. Computer industry news: links to articles about the computer industry.

      Rating: 3

   5. Personalized information: a computer book web site that provides you with personalized information about its products based on your past purchases or profile information you provide.

      Rating: 2

   6. Downloads: free software you can download off the Internet (shareware)

      Rating: 4

   7. Instant title notification: email notification each time a new computer book title is published.

      Rating: 1

   8. Monthly email newsletter: announcing new titles for the month.

      Rating: 1

   9. Online books: computer books you can access and read from the computer book web site.

      Rating: 5

   10. Online chats

       Rating: 2

   11. Online purchasing: the ability to buy a book directly from the computer book web site.

       Rating: 5

   12. Online registration: an online form you can use to register your book with the publisher.

       Rating: 1

   13. Online training courses: for-fee computer courses offered through the computer book web site.

       Rating: 1

   14. Press releases: news about the company and new book titles.

       Rating: 2

   15. Professional book reviews: reviews that were previously published in computer magazines.

       Rating: 4

   16. Reader reviews: comments from other customers who purchased the book.

       Rating: 3

   17. Specials: discounts off selected computer book titles or sets of books.

       Rating: 3

   18. Technical job listings

       Rating: 1

   19. Tips: for using a computer program

       Rating: 4

   20. Upcoming books: a list of new releases expected during the next month.

       Rating: 1

2. Which features or services do you consider to be the most helpful? Please insert the letter of each item you select from the list above. You may list up to five items.
   1. i
   2. o
   3. b
   4. f
   5. k

3. Which features or services do you consider to be the least helpful? Please insert the letter of each item you select from the list above. You may list up to five items.
   1. a
   2. g
   3. l
   4. t
   5. j