Deciding Research Questions

As novice researchers, we wanted to investigate how any previous focus of VLE enquiry might best be transferred to evaluate courses. Having turned to the literature, we embarked upon an approach to building our research criteria from first principles. Dempster (2004)[1] suggests that for individual innovators, a pragmatic approach is to identify what aspects to know about your use of e-learning, the impact on students’ learning and their experience of learning. The next step is to formulate the key questions that need answering. Questions should relate back to your teaching goals and the overall context of learning while taking account of others who might learn from your experience. “If you don't have a question, you don't know what to do (to observe or measure); if you do, then that tells you how to design the study” (Draper, 1996).

This then was how our basic questions for the evaluation of Ikarus were developed. We were interested to know to what extent Ikarus provided a learning place:

a.    as a meeting place to communicate,
b.    to share new ideas,
c.    to experience learning

It could be argued that the above questions are too broad. To narrow the focus, we realised we needed to be more specific. In order for our focus to be more in-depth we thought perhaps we could explore how well Ikarus might meet the key tenets of Chickering & Gamson’s ‘Seven Principles for Good Practice in Undergraduate Education’[2].

The 7 Principles were developed as a means of evaluating undergraduate education and although they were first used almost twenty years ago; our rationale was that learners’ needs in HEIs were still consistent today. Chickering & Gamson’s 7 Principles are below:

1. Encourages contacts between students and faculty.
2. Develops reciprocity and cooperation among students.
3. Uses active learning techniques.
4. Gives prompt feedback.
5. Emphasizes time on task.
6. Communicates high expectations.
7. Respects diverse talents and ways of learning.
 
Our wish to conduct deep & highly-focussed qualitative research had been a reoccurring theme throughout WS3. For example, ways of framing a question that would meet the requirements of 2 above could be:

“In what ways that were developed by the learning community, did the design of Ikarus succeed in constructing a shared understanding in the individual learners?”

“In what aspects did learners feel that their learning was being facilitated by their co-learners?”

“In what aspects did learners feel that expectations were being met?”

“In what ways did the design fail to meet the aspirations of the participant learner?”

“To what extent can supportive messages within bulletin boards encourage further discussion & deepen reflective learning?”
 
Further questions were drawn from current thinking on evaluating LAMS[3] whilst not directly relevant to VLEs, some amount of cross-fertilization was a possibility; hence current discussion about evaluating LAMS was considered viable as a spring-board to generate questions of enquiry that could be used for Ikarus. As a consultant to the JISC e-Learning and Pedagogies Programme, Beetham (2004)[4] suggests the following questions:

Activity - we know this fits with the current theoretical literature on howpeople learn, but how far is the current focus on 'activity' rhetorical? i.e. what real differences do the new tools and affordances make tolearners, as compared with tools that appear to focus on learning 'objects'? What notion of 'activity' do these tools and specifications encode? Does this fit with the notion of 'activity' held by learners, teachers, and educational theorists?

Sequence - to what extent does sequence or workflow uniquely define alearning activity? What other factors need to be taken into account? Whenand why is sequence highly significant to learning outcomes?

Approach/subject - what, if any, is the relationship between the types of activity and sequence chosen and the pedagogical approach (and/ordiscipline/subject context)? Are particular sequences or activities moreeffective in certain pedagogical and/or subject contexts?
Interactivity - does the focus on activity lead to more peer interaction? More equitable contributions from learners in groups? Higher levels ofparticipation and motivation?

Sharing and re-use - How re-usable are 're-usable' activities and/orsequences in practice? What contextualising information is necessary foreffective re-use? What is it that is actually re-used (i.e. what is a'learning design')? What evidence is there for re-use across subject and sector boundaries?

Designing for learning - How do practitioners currently plan, design and orchestrate learning activities? How much time is allocated for this? Whatskills do they need? How much discussion do they have with colleagues? What are their constraints? What software, paper-based and other tools support this activity at present? What tools could support this activity more effectively? What tools and approaches might 'disrupt' and develop this activity?

The learning design specification - What 'fit' does this offer with existing practitioner conceptions of activity, sequence, approach and design as identified above? What potential does it have for transformation of practitioner conceptions? Could this transformation be justified in terms of (a) learner outcomes (b) ease of sharing and re-use (c) development of more effective tools and teaching systems?

[1] Dempster, J. (2004) http://www2.warwick.ac.uk/services/cap/resources/eguides/evaluation/elearning/questions
 
[2] Chickering, A.W. & Gamson, Z.F. (1987) ‘Seven Principles for Good Practice in Undergraduate Education’  http://honolulu.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/7princip.htm   
[3] Learning Activity Management System (LAMS) is a revolutionary new tool for designing, managing and delivering online collaborative learning activities. It provides teachers with a highly intuitive visual authoring environment for creating sequences of learning activities including individual tasks, small group work and whole class activities based on both content and collaboration. http://www.lamsinternational.com/index.html
 
[4] Beetham, H. (2004) LAMS Research Questions published via LT-THEORY@jiscmail.ac.uk (21/06/04)