Yannis Karaliotas
MA in ODE
H801
July 2000

Inquiring on the Web:
Evaluation of Web-based Searching

This paper reports on a simulated web-search exercise. Within the boundaries of this exercise, the students/practitioners of the MA in ODE course H801/2000 were called upon to compile and evaluate DE course information, related to their particular subject area, using Web-based searches. The paper is divided into three parts:
Part 1 illustrates the collective processes of performing the set task by reporting on shared peer reflections as they appeared in the relevant (online) course-discussion.
Part 2 elaborates on the writer’s personal case study – i.e. evaluation of three relevant web-resources using appropriate criteria.
Part 3 evaluates the methods of obtaining information from the Web as opposed to more traditional methods.

Part 1 :

- The Scenario - Having been assigned with the evaluation of distance-taught courses in a given subject area in order to “report on the potential for new courses”, the ODL practitioner is entrusted with the task of collecting and critically evaluating information which would enable the institution/organisation to “make a practical decision about which courses (if any) it should offer in this area” [ App. 1 ] & [ App. 2 ].

- Task Analysis - The task in hand appears to have two essential requirements:

  1. The construction of a well thought set of explicit criteria which would enable the researcher to
  1. identify research requirements (i.e. the kinds of information s/he is looking for)
  2. construct appropriate strategies which would best exploit the properties of the medium for a purposeful search
  3. critically evaluate her/his findings.
  1. The widest possible pool of current, recently updated resources where relevant information can be drawn from.

The predetermined choice of the Web, which is indeed a searchable, constantly updated pool of almost unlimited resources, seems to have satisfied the latter requirement. Hence, as could be expected, the over-a-month long peer discussions in all four working groups focused on criteria and methods for strategic searching and convincing evaluation .

- Fundamental concerns and the prioritisation of values - First, clarifications were needed concerning the aims of the search:

What is the policy to be directed at, for example:

Also, advice was sought as to the way of reporting on findings:

“ [...] how can we relate ‘research’, ‘scholarship’, ‘reasoned argument’, ‘polemic’, ‘rhetoric’ etc with material on course descriptions? The course material that we search for on the web surely is not research material, but course outlines? ” [ nc164 ]

Peer exchanges regarding fundamental what, why and how questions repeatedly appeared in all four conferences until the end of the discussion, as peers reformed and refocused initial understandings [ App. 3 ]. Delving into the issue of having to justify academic/pedagogic values vis-à-vis managerial requirements, the imposed condition of business oriented criteria [ App. 4 ] that tend to commoditise education was questioned:

“ I find Martin's claim pragmatically correct when he says that : "Basically the policy document to managers has to “speak the language of managers” to win approval and I’m not convinced my managers are really interested in any pedagogical aspects!" [ msg#191 ]. However, I don't feel that our assignments, or even our actual policy documents for that matter, should follow the semantics of business managerial discourse and be empty of academic argument (Martin, msg#192 & Eva, msg#195 ).

[...] it is the duty of educators to question (business) semantics (i.e. meanings attributed to words in discourse) - generating rather than imitating elements of discourse.” [ nc199 ]

 

- Searching the Web -

“I am still trying to get over what I see as the biggest hurdle, namely how to do an EFFECTIVE search rather than a scattergun.

How does one develop an effective search strategy?” [ nc197 ]

The Internet and its Web with their practically infinite storage capacity has become the ideal place for obtaining and/or retrieving information. However, searching the Web requires knowledge and use of electronic search tools . These are usually Web-based machines, linked to appropriately storage-spaced databases, programmed to search specific domains or the entire Web according to the commands given by humans through software-driven human/machine interfaces.

The bulk of peer exchanges referred to the subject of Web search engines by:

After initial trials, and according to the level of familiarity, frustrations with spending too much time web searching and getting information overload , concerns with issues of credibility and verification , and the need for a strategic approach involving critical reflection on refining criteria [ App. 5 ] were reported:

“I'm fairly new to computers so I began with a guide to searching the Web [...] This told me (I think) that there are 3 kinds of search engine - search directories, search indexes and search both (?metasearch).

So I did an index search [...] Then I did a metasearch [...]

Summary[of] main problems: time consuming, scant relevance, limited perspectives on evaluation, insufficient detail, inaccuracy. Information gained would only enable me to report an outline of distance [...] courses advertised on the Web.

Any tips on refining searches most welcome ....” [ f276 ]

“The problem with this was that a search on [...] delivered information overload, which if I had a year in which to do this assignment might not be an issue. And so it was back to some critical reflection on what I wanted to research on and to redefine my terms and the boundaries of the search. [....] And so, back to the criteria and refining the search. And if you have not yet started, my advice would be to carefully define your boundaries of what it is you require and what will be relevant and useful to you, keeping focussed on the questions in the assignment.” [ f275 ]

 

- Direct Human Synergy – Due to the emerging problems, the need for sharing insights became evident. Notwithstanding scale differences, this naturally unfolding, intersubjectively collective, iterative peer-collaborative process of inquiry generated a pattern that seemingly fits the schema of Fourth Generation Evaluation – The Methodology of Constructivist Inquiry ( Cuba & Lincoln, 1989 ).

Figure 1 : The Methodology of Constructivist Inquiry ( Cuba & Lincoln, 1989 )

Having based the task of inquiry upon, both, the collective ( learning community concerns) and individual (subject concerns) real needs of H801 peerhood, a natural setting within which the task would be conducted was enabled.

The use of human instrument and qualitative methods is naturally embedded in collaborative learning environments where knowledge is constructed mostly through learning transactions among peers and “[c]ontexts give life to and are given life by the constructions that are held by the people in them” ( Cuba & Lincoln, ibid. ). However, going deeper along these lines, the a priori condition for human synergy and empathy in a peer-collaborative environment (H801 virtual learning environment) unveiled the possibilities of human agency and human networking in a much wider inquiry setting (the global networked community):

“ [...] Another point of departure is posting questions to mailing lists and/or searching their archives – even general DL-interest lists would do. This way you make use of human agency, not just bot.com technology. [...]” [ nc169 ]

“ [...]Don't reinvent the wheel. Where possible use other credible lists and reference sites to get the information you want. [...]” [ nc196 ]

“ [...]However, I think one of the lessons I'm learning is that no matter what you do, the need for human intervention always prevails. [...]” [ nc173 ]

“ [...]Aren't humans wonderful! Back to the inhuman search engines!” [ f190 ]

Likewise, the use of tacit knowledge for metacognitive strategies and lateral thinking approaches in conducting Web research appears to be indispensable:

“ The challenge (fun and frustration) I find is deciding when to chase a 'prey' in the hope it produces a meal, or simply ignore it and look elsewhere. The discussions we are having remind me of some rather basic animal instincts on whether to expend limited energy resources on looking hard for supper, or wait a while for somebody to tell me where an easy prey is waiting unguarded! ” [ nc204 ]

- Vicarious experience and the construction of ‘goodness criteria’ – Throughout the discussion, peers acknowledged the vast capacity of the Web in providing the inquirer with a plethora of resources. However, because of the vastness of resources and the limitations of search-bot technologies, it became evident that vicarious experience , gained through the construction and reconstruction of shared subjective understandings, is crucial for informing contextualised goodness criteria for purposeful and convincing research. There are certainly lessons to be learned from this Web-search project:

“ [...] I guess most important thing to come out of this for me is the value of human agency/ interaction (which seems like a motherhood issue here), even for more passive players like myself, when you need to find out what's really of value in any arena of common interest. The story generated by spiders and indexers and other automatons follows one sort of logic (i.e. all bits and bytes are created equal), and the story generated by subject editors at major dotcom sites follows logic of another sort (i.e. value is ultimately measured in monetary terms). Be all that as it may, to a certain extent, be useful, I still don't know of a better way to get real insight as to where value lies in a subject area, than inside a community of interest/ practice like this one. [...]” [ nc228 ]

 

Part 2 :

- Personal Case Study – The School of Education of a Hellenic HE institution explores the potential of offering flexible learning courses in Educational Technology to trainee teachers, as part of their practicum, and to teachers-in-service as professional development or postgraduate modules. Hence, the selection criteria for resources sought on the Web must correspond to the DE views and objectives of a Hellenic HE institution and the perceived needs of its learners.

According to Rowntree (1999) , the implementation of distance learning is not a matter of uniform practice but, on the contrary, requires serious context analysis so that decisions would correspond to real needs. Peer exchange in the H801 multicultural learning environment helped to better identify the similarities and differences that exist among different national systems of education and different institutions. The public nature of the publicly funded Hellenic HE system, safeguarded by the state constitution, cannot reasonably accommodate practices of market competition or notions of profit making.

The use of a non-internationalised, non-Latin scribed language which precludes the expansion of an Hellenic educational industry beyond the national borders, and the absence of a sizeable market which could seriously attract the attention of foreign DE industry are important factors contributing to resisting imitation of North American practices or outright adoption of recent W. European trends in HE. Despite shared concerns about maintaining quality in education while making an effort to satisfy economic mandates for reducing public expenditure, the perspectives of Hellenic academic practice appear to be markedly different in that they are not profit driven.

Hence, the search for Web resources on the subject of Educational Technology DE courses was made with the understanding that the inquiry aimed at exemplary practices and quality, cost effective ideas that could help to improve teaching-learning practices and enhance subject development, rather than at evaluating the competition. It seems important to define the value system that underlies institutional and life decisions; problems occur when one set of values, be they economic or academic, begins to eclipse the other, to the detriment of both. One needs funds to run a school, but a school whose primary focus is to make a profit might begin to cut corners!

 

- Goodness Criteria

“We start with particular cases of knowledge and then from those we generalise and formulate criteria of goodness - criteria

telling us what it is for a belief to be epistemologically respectable.” ( Chisholm, 1982 )

The search aimed at obtaining information from the Web on distance or online taught or supported courses in educational technology , technology in education , learning technology or mathetics within teacher training , in-service training , professional development or postgraduate programmes offered by HE institutions or dotcom entrepreneurs.

Mailing lists (archives of and/or postings to DEOS, ITForum, IFETS, DUC, dldc, EDUlist(gr) & SOIS(gr) listservs), bot engines (Google™, WebFerret™, Copernic™ & FindIN.gr), and direct peer to peer contact (e-mail) were used as the means for conducting the search. Initial selection from bot engine findings was performed offline in a ‘purposive sampling’ manner ( Cuba & Lincoln, ibid. ), using the Find function of Windows™ Explorer, under the following initial selection criteria [ App. 6 ] :

  1. priority to commercial, Australian and European domains (.com, .au, .uk, .gr etc.) – hard to find among US .edu links
  2. search for master key words (e.g. course or online ) in link descriptions
  3. search for words/phrases in individual descriptions that indicate possible strong association (either positive or negative) to ‘goodness’ criteria - use of tacit knowledge .

These selected bot findings, along with the ones derived from the other two sources, made up a sample base of 87 items for further processing. Looking at the selected resources in detail, an open list of goodness criteria was constructed under the headings:

  1. quality of content and delivery
  2. appropriate use of media/technology
  3. quality of learner support
  4. cost effectiveness

A general observation has been that, although there is a considerable number of Educational Technology related courses offered at a distance, these are recently introduced and cannot offer many historical insights as to their development and status.

The resources to be evaluated using the above criteria were chosen as most representative of the principles laid in the initial selection phase and indicative of contemporary DE trends in both national and global context. ( Table 1 )

 

Documents

 

Evaluation

 

ED552 : Models of Teaching, Learning and Technology

As Stand-alone (self-contained) or programme module

OSU Web Course

(USA)

 

  • Content & Delivery: Technology and technological change in contexts of learning theories. Combined theory-practice approach directed at enhancing the learner's understanding of the relationship between models of teaching, learning theory, and technology. Knowledge base (resources) connects technological innovations in a global scale.

Flexible structure, localised delivery – CMC supported home-study with classroom-based constructivist activity preparation and implementation.

Project oriented and CMC dialogue content assessment.

  • Media/Technology: mixed, integrated use – print material and web resources.
  • Learner Support: Tutor and peer online support, and additional mentors programme in a learning community environment .

Accessibility – despite strong mission statement, there is no evidence of off-campus disability support (e.g. Braille transcripts or taped versions of print materials).

  • Cost Effectiveness: Low production cost – easily updated material with possibilities for recombining and recycling.

Learning Schools Programme on ICT use

Open University and RM Professional Development commercial package

(UK)

  • Content & Delivery: There is little on site access to content information. However, there seems to be a connection between this commercial package and OU courses E850 & E851 on offer for the first time in 2000 and 2001, respectively.

Practical orientation – material brings together case studies for skill development in the use of ICT as teaching tool and practical ideas and approaches to cover individual needs and skills levels for serving primary and secondary teachers and librarians.

Focused structure, localised delivery – school-based self-study with classroom-based activities.

Project oriented assessment .

  • Media/Technology: mixed, integrated use - printed documents, multimedia CD-ROMs, BBC produced videos and use of web-based materials.
  • Learner Support: Only technical support on site – if OU E850, E851 support applies, then, use of CMC for tutor and peer support and possibility of learning community building is provided.

Accessibility – strong OU support on alternative delivery of materials - though not ready for year 2000.

  • Cost Effectiveness: Costly multimedia production, but possibilities of simultaneous use and reuse of learning modules (objects) in other programmes. Low cost of support as it allows for part-time staffed continuous asynchronous support, provided both ends are network connected.

 

TEA Course Telematics and Distance Education (in Greek)

University of Patras teacher training module (GR)

 

  • Content & Delivery: Resource base approach – material gathered on site selected from a much smaller than global knowledge base (language), necessitating translation and adaptation of additional Anglo-scribed resources.

Theory oriented material for postgraduate secondary math teachers.

Mixed delivery – online to support f2f teaching.

Personal initiative.

Project oriented assessment.

  • Media/Technology: On campus delivery with the use of web resources and CMC facility with the use of a dedicated listserv .

Accessibility – No policy declared, but course on site material (text) and CMC tool accessible.

  • Learner Support: On campus tutor support and attempt to facilitate peer support through the use of mailing list.
  • Cost Effectiveness: Modularization of resources facilitating recycling of materials among different courses.

Table 1 : Evaluation of Educational Technology related DE courses

- Conclusions – National HE needs and requisites regarding the development of DE teacher education courses in Educational Technology and Mathetics might be better served with the use of flexible , mixed delivery of combined theory and practice oriented material , utilising networked CMC learning environments where online tutor and peer learner support is facilitated and alternative learning assessment methods are enabled.

The mobilisation of part-time tutoring staff from already existing human resources, the launching of mentoring programme incentive initiatives and careful preparation of web oriented modularised material (learning objects) may help to maintain low costs of delivery. These attributes introduce necessary conditions for provision of effective continuous learner support through the use of already existing asynchronous online (on- and off-campus) facilities. The above schema may also provide for low cost solutions to accessibility issues.

One could reasonably generalise that modularization of well balanced (theory-practice) relevant materials ( hypertext/hypermedia oriented modularity ), a low cost networked (Web) interactive learning environment , where knowledge construction through human learning interaction is strongly encouraged, and the use of alternative, construction oriented, assessment methods are the main attributes of potentially successful distance courses, even in mixed delivery modes ( Karaliotas, 2000 ).

Part 3 :

“I think the web can offer us a fantastic amount of high quality information but the price that we have to pay for using this information is that we all have to become smarter searchers and a more critical audience.” [ d680 ]

Naturally, there are pros and cons in the use of the new media and technologies of the Web for research. Perhaps, in attempting to evaluate obtaining information from the Web as opposed to more traditional methods, an overarching consideration may be the fact that most credible traditional resources are significantly outdated as compared to the immediacy and update-ability of web resources. This is a nontrivial observation considering the rapid development and breathtaking paradigm shifts occurring in all cognitive domains.

There is, certainly, the difficulty of coping with the “sheer size of the web” and “the vast quantity of stuff” [ d639 ]. One has to deal with typographical mistakes [ nb179 ], inaccuracies [ f276 ], outdated [ nb174 ] or redundant information, connection problems, special software requirements for downloading of information, restricted access (password protection) and costly telephone bills [ d638 ]. The Internet is, indeed, a non-linear, quite formless, digressive medium which poses new demands and requires new skills [ nc210 ]. But, also, opens new ways and offers new opportunities for purposeful and convincing research.

Frequent questions that often emerge in relation to obtaining information from the Web concern the criteria used for judging the quality of textual analysis. These are often relate to the positivist notions of ‘truth’ and the conventional benchmarks of ‘rigor’, namely

which, in turn, regulate the concept of credibility and relevance. It is obvious that the dynamic, organically evolving Web cannot easily accommodate such rigid ‘quality criteria’. But are these criteria consistent with the image and substance of contemporary research?

This conventional framework is being increasingly replaced under the view that reality is a social and, therefore, multiple construction and the "knower and known are interactive, inseparable" ( Lincoln & Guba, 1985, p.37 ). The subject-object dualism is replaced by a subject-subject relationship, and a collapse of the distinction between explanation (auslegung) and interpretation (erklaren). There is therefore a celebration that "inquiry is value-bound . . . an opportunity to be exploited" ( Lincoln & Guba, ibid., p.37 & p.101 ). Truth in constructivism is an equivocal term, having as many meanings as there are procedures of justification. This involves a move away from generalisability to transferability in answer to the challenge of knowledge accumulation.

Constructivist criteria for judging the quality of textual analysis stress the norms that guide the social practice of dialogue and negotiation - judgements are dialogical so the crucial factor is the character of the social interaction dialogue. The regulative ideal of constructivism is therefore solidarity and synergy . The usual positivist criteria can be replaced by trustworthiness terms such as credibility , dependability , confirmability , and, as noted above, transferability ( Lincoln & Guba, ibid. ), or authenticity criteria of fairness such as "ontological authenticity (enlarges personal constructions), educative authenticity (leads to improved understanding of constructions of others), catalytic authenticity (stimulates to action), and tactical authenticity (empowers action)" ( Denzin & Lincoln, 1994 ).

This alternative framework raises the importance of ‘grey’ literature (Harry, 2000 ) in research, and better fits the image of the contemporary researcher as the ‘transformative intellectual’ - an advocate and an activist - and as the ‘passionate participant’ - a facilitator of multi-voice reconstruction-, rather than the one of the ‘disinterested scientist’ as informer of decision makers, policy makers, and change agents. The organic nature of the Web enables and strengthens rather than inhibits purposeful and effective research performed in that light.

- The meaning of ‘traditional’ – There has to be an objection that setting the comparison between obtaining information from the Web and using more traditional methods may be a bit misleading as to the methods and means (e.g. there is nothing stopping you from writing (email) directly to the chosen source if you feel you need to). Perhaps, comparing the act of information seeking at a distance using web technology (which includes even synchronous means of communication) with direct human contact in inquiring might be more appropriate.

Along these lines, what seems to be making the Web an indispensable tool for conducting research is its hypertextual ability to instantaneously and persistently network human thought. If this is harnessed through the use of e-tools which enable and promote human agency and direct contact (e-mail, mailing lists, asynchronous and synchronous text-audio-video conferences, MOOs, BBSs etc.), this would prove, I feel, the Web way of bridging the gap between machine linear feed and the dynamic nature of direct human inquiry [ App. 7 ].

 

References

Chisholm, R. (1982). The Foundations of Knowing. Minneapolis: University of Minnesota Press.

Denzin, N. K. & Lincoln, Y. S. (1994). Handbook of Qualitative Research. London: Sage.

Guba, E. & Lincoln, Y. (1989). Fourth Generation Evaluation. London: Sage.

Harry, K. (2000). Introducing the Literature: Sources and Characteristics – H801 Online Study Guide 2000, Block 3, Section 2. H801 material.

Karaliotas, I. (2000). Teaching at a Distance: Improving Students’ Learning - Advances in the teaching/learning debate. MA in ODE Student Assignment. Online source: http://users.otenet.gr/~kar1125/tld.html

Lincoln, Y., & Guba, E. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage.

Rowntree, D. (1999). Making open and distance learning work - An overview essay with hypertext appendix. H804 material. Online source:

http://www-iet.open.ac.uk/pp/D.G.F.Rowntree/MOADLW.htm

 

 

Appendix

(H801 Assignment Guide 2000 – SUP 47073 4)

Imagine that your institution or organisation is considering introducing new courses in your own subject area. You have been asked to prepare a report which evaluates distance-taught courses in this area.

Part 1: Use Web-based searches to collect information for this proposed report. Discuss with your H801 colleagues in the TMA Workshop for your tutor group how you might set about this exercise, including the search engines you would use and how you would refine your search to obtain relevant material. Tell them the problems that you encounter in your search. Then, for Part 1, report the discussion in your own words (not more than 1500), indicating your own contributions to it.

Part 2: Which explicit criteria would you use to evaluate material that you have collected. Apply these criteria to three of the documents and write up your findings.

Part 3: What do you see as the strengths and weaknesses of obtaining information from the Web as opposed to more traditional methods, such as writing directly to institutions?

----------------------------------------- UP

(2) TMA04 guidance Hi everyone. Here's the promised advice on TMA04. Before I get into the detail I would just like to remind you that you do need to make use of (and cite) relevant course materials, in particular the Block 3 resources which link you to other relevant sites. Among other things, this should help when you are thinking about the criteria for evaluation you are adopting. I will flag some of the relevant materials below.

TMA04 invites you to use the Web to collect information for a proposed report on the potential for new courses in your subject area. It's clear from the first paragraph of the TMA question that you are being asked to prepare a policy document, not a piece of academic research. You are essentially identifying resources which will help you to evaluate the competition, to find out what courses are on offer in this area, what they contain, how they are taught, the context they operate in, their target audiences and so on. So the information you gather would be needed to help your institution make a practical decision about which courses (if any) it should offer in this area.

Part 1 asks you to conduct Web-based searches to collect information for this proposed report. In order to do this you need to consider what kind of information you are searching for and where it is likely to be. You need to find out as much as you can about course offerings in this area, so just looking at the Web sites or online prospectuses of rival institutions will not tell you all you need to know. Different kinds of information, and different types of document, are likely to inform your decision. So before you search, think about the task your institution has set you.

a) What kinds of information do you need to evaluate rival courses? In other words, decide on your game plan before you start searching. What would you really need to know to help you decide whether to launch a new course/courses? Who or what is likely to have that information? How might it be organised and/or made available? How would you start to look for it? Do you need to inform your institution's management of wider professional, pedagogical or policy contexts in order to help them make a decision? You might like to discuss this amongst yourselves online.

b) Having identified your information needs, you then need to define a Web search strategy. Which search engines might you use, and why? What tools do they offer for refining a search? What search terms would you use? Search engines have limitations - they each have their own strengths and weaknesses. So what other Web-based resources do you know about which might have the information you need? We have provided you with some online resources on the course Web site but lateral thinking is recommended here. Again, you might want to share information about search tools, resources, strategies and the kinds of information you are searching for with your peers in the TMA Workshop. You are also encouraged to continue the discussion by reporting problems and issues that arise as you do your actual searching.

Part 2 asks which explicit criteria you would use to evaluate material that you have collected. This is a matter for your own judgement, but clearly you will use your own professional and academic experience to assess what you have found. You also need to think about what your institution's priorities are, because your report would also need to meet the institution's evaluation criteria. Although there is doubtless overlap, your own priorities as a practitioner and your institution's priorities are unlikely to be identical.

For Part 2 I think that Section 4 of the study guide is particularly relevant. Of course you are not being asked to look for research papers here. Nonetheless, Section 4 provides examples of ways of using criteria, which can be critically modified. If you prepare for Part 2 by looking at these you will probably be better off than if you were to start thinking about criteria from scratch.

Part 3 seems clear enough - you are asked to assess the strengths and weaknesses of the Web for gathering the information you need as opposed to traditional methods. Although the question appears straightforward there's a lot that can be said on this issue I think. You may find Section 6, which looks primarily at sources in print as opposed to on-line useful for your Part 3 answer.

Nigel
------------------------------------- UP

(3) Indicative of discourse, the statistics of relevant terms appearing in about 400 messages are as follows:

'policy' : 42
'quality' : 35
'standards' : 10
'values' : 10
'strateg' (root) : 30
'compet' (root) : 35
'market' : 30
'business' : 20
'commercial' : 15
'money' : 10
'profit' : 12
'fund' (root) : 21
'pedag' (root) : 23
'academ' (root) : 36

[Search performed on H/D with 'Find' function of Windows ™ Explorer]
---------------------------------------- UP

(4) [...] TMA04 invites you to use the Web to collect information for a proposed report on the potential for new courses in your subject area. It's clear from the first paragraph of the TMA question that you are being asked to prepare a policy document, not a piece of academic research. You are essentially identifying resources which will help you to evaluate the competition, to find out what courses are on offer in this area, what they contain, how they are taught, the context they operate in, their target audiences and so on.

--------------------------------------- UP

(5) Knowledge, Skills and TMA04.. [d659]

Below is partly about researching being an Art and not a Science (as some 'step by step' courses on searching seem to suggest….). And I see TMA04 within Block 3 as 'development of the Research Arts'. Or have I missed the point (again!)?

I think that the starting point in Part 1 is not only the specification of the objective but also evaluation criteria (and I think they work in Part 2, though with some changes of focus). At first I was a bit stunned by David's ref. in the TMA04 advice above that evaluation criteria could be as limited as 3 but, after some re-thought realised that I was working with a basic 3 of:

? Relevant
? Complete
? Accurate

Also, I think that you're working back and forth between different 'levels of evaluation insight' as you look for/accept/store/reject information.

1. Knowledge of your own specialist subject area. Key players, 'state of DE', professional associations/listserves, reputations in the field, personal contacts, print sources of info. etc.

2. Knowledge of the wider domain. (e.g. as of DE which we get through H801)

3. Knowledge of where to find 'leads' to information outside your personal knowledge (as in the leads given in Block 3, or us as potential mines of information).

4. Skill in using available resources e.g. libraries, where/how to search the Web, the validity of different types of information -- even how to approach institutions personally for information.

So you could say that Part 1 of the TMA is about Critical Thinking creating a set of evaluative criteria. As you go you modify using Critical Reflection according to the results and some Creative Thinking:

Is it that the info. doesn't exist?
Is outside the cost/benefit criteria?
Could it be found more efficiently through a different route?

Is there anything that I can do about my personal knowledge/skills to improve the results?

I know I'm trying to make it 'cut and dried' and fit the logic into boxes when, in fact, it's such a complex thing. That's where I'm finding the problem with the TMA. It will not fit into the 'Parts'. It's just not that sequential. However, that's a valid conclusion in itself, perhaps?

Slightly different bit on Some Skills I Have Gained:

I've realised that I am actually rotten at storing what I have found or structuring how I find information. What I have learnt, skillswise, is :

? That depending on my memory is really not enough. This might be a 'post-40' thing as I could swear that it used to be better…It's worth the time not to panic about the being online costs and to keep a short record as you go of what led to what.
? I'm running 2 browsers for the first time, so that if it's not in a 'frame' and I think it will be a side-track, I put it into the other browser and mess about there.
? I bookmark into files the things that are 'only potentially interesting' but copy to a Word document file on things that are important.
? If I can't get the URL into the 'Word' title I now make very sure that I copy it into the head of the page.
? I have a 'return' bookmark file for those things that won't download quickly enough but are potentially interesting which I check and delete every few days.
? I consciously refuse to be side-tracked.
? I have fallen in love with Copernic because it stores the searches and I can delete down, tick the 'finds' that are really useful and leave the ones which merely prove interesting and can return back – and it's coded.
? I've started to actually provide feedback to sites where something is not accessible or appears to go wrong. And they e-mail back with a 'thank you' and e.g. the correct link!
? I print the 'home' or relevant page of anything important so that if all else fails I can get back to it and I write at the top where it came from/what it refers to. I then file it instead of putting it on the floor!

Organisation seems to be the name of the game. What am I doing that is pointless or can be done much better? If you have a personal approach that you don't want to put into the EBBS, e-mail, I beg.

--------------------------------------------------- UP

(6) .....working back and forth between different 'levels of evaluation insight' as you look for/accept/store/reject information.

1. Knowledge of your own specialist subject area. Key players, 'state of DE', professional associations/listserves, reputations in the field, personal contacts, print sources of info. etc.

2. Knowledge of the wider domain. (e.g. as of DE which we get through H801)

3. Knowledge of where to find 'leads' to information outside your personal knowledge (as in the leads given in Block 3, or us as potential mines of information).

4. Skill in using available resources e.g. libraries, where/how to search the Web, the validity of different types of information -- even how to approach institutions personally for information. [d659]

--------------------------------------------------- UP

(7) [...] Also, different topic, can I put in a big, big + for computer accessed info. here. Many students – like me - have no library access at all so a course would be plain impossible if it required access to hard copy texts or information. I also move from country to country and within the country. I know that other students on H80* are similarly unpredictably mobile. The transportable any time/any place nature of Web resources is very important to peripatetic (is that the right word?) students. I imagine, conversely, to those who, for whatever reason, are house-bound.


URL: http://users.otenet.gr/~kar1125/erevna.html
Copyright © 2000 Ioannis (Yannis) Karaliotas