Thinking about a possible Part 4 for the questionnaire -
- (2009-03-04) I numbered the questions to refer to them in a concept map.
- (2009-03-06) Copied Jenny's table from her public blog post here and numbered the statements.
Part 4 could be entirely separate from the first parts or could be a continuation. My rational for this part is to go beyond collecting simple quantitative data and to begin to explore people’s communication and learning experiences on the CCK08 course, focussing on the differences between those who preferred blogging to discussion forums and vice versa.
So key research question is?
Why did some CCK08 participants choose to blog rather than post to the Moodle forums and vice versa?
Sub questions might be?
Did these two groups of people have identifiable group specific common characteristics?
What are the advantages and disadvantages of blogs and discussion forums for self-directed learning?
I have remembered Soini and Flynn’s article in which they identify 6 aspects of critical learning instances (see notes)
Emotional significance (62%)
Reflection (53%)
Differing perspectives (42%)
Collaboration (38%)
Autonomy (33%)
Dialogue (24%)
It occurred to me that Part 4 could include sets of statements based on these or a similar adapted list and a Likert Scale type of questionnaire, i.e. Strongly agree to strongly disagree.
The following section is about blogging and forum discussion. Please read each statement carefully and indicate to what extent you agree or disagree using the folloiwng key:
strongly disagree neutral agree strongly
disagree agree
1 2 3 4 5
or
SA - Strongly agree
A - Agree
N - Neutral
D - Disagree
SD - Strongly disagree
Example statements: SA A N D SD
1. My firm is committed to achieving zero defect logistical performance O O O O O
2. My firm has different logistics service strategies for different customers O O O O O
3. The orientation of my firm has shifted from managing function to manaing processesO O O O O
OR with the SD, D, N, A, SA (denoting 1, 2, 3, 4, 5) SD D N A SA
1. My firm has increased inventoy turns during the past 5 years O O O O O
For Relationships
IMPORTANCE
(Please circle the number under the answer which is right for you) (or this could be done using survey monkey)
Important Of Very Of Some Important Very Important
Not to have Little or No Importance To have To have
this Importance To have this this
this this
1 2 3 4 5
Example statements:
1. Respect you
2. Are friendly
3. Are good at their jobs
4. Work together well
SATISFACTION
Very In Some Very
Dis-satisfied Satisfied ways Satisfied Satisfied Satisfied
Some ways
dissatisfied
1 2 3 4 5
I found different researchers have been using different scales for their researches: such as 1 very dissatisfied to 5 very satisfied, OR 1 very satisfied to 5 very dissatisfied. I think we need to be consistent throughout the questionnaire.
May I suggest to adopt a lower score such as 1 for very dissatisfied and a higher score of 5 for very satisfied? We could use that to calculate the average satisfaction rating and the variance (or Standard Deviation) using those scores. Similar concepts could be applied to the importance ratings in the questionnaire.
For all questions - check if they are written in SIMPLE PLAIN CONCISE ENGLISH John 6/3/09
An alternative way of doing this would be two have two statements at each end of a continuum, e.g.
There was a greater sense of community between CCK08 forum participants
|
0
|
2.5
|
5.0
|
7.5
|
10
|
There was a greater sense of community between CCK08 bloggers
|
Participants would then mark their position on the continuum
Or we could go for a Likert Scale, strongly agree to strongly disagree, using the following types of statements
Which of the following is your preferred tool of learning and communication in CCK08?.
1. Blogging
2. Moodle discussion
3. No preference (Blogging/Moodle discussion of equal importance)
Or should we separate learning from communication? John 6/3/09
Experiences related to technology
- 2 Posting to the CCK08 Moodle forum required less effort than posting to a blog
- 3 Posting to the CCK08 Moodle forum enabled faster communication than posting to a blog
- 4 It was easier to follow a CCK08 Moodle forum thread than a blog conversation
Emotional significance
- 6 There was a greater sense of community between CCK08 Moodle forum participants than between CCK08 bloggers
- 7 There was less mutual respect between the CCK08 forum participants than between the CCK08 bloggers
- 8 Relationships between the CCK08 Moodle forum participants were stronger than between CCK08 bloggers
- 9 The CCK08 Moodle forum posts elicited stronger emotions than the CCK08 blog posts
- 10 Posting to the CCK09 Moodle forums generated greater feelings of excitement than posting to a CCK08 blog
- 11 Posting to the CCK08 Moodle forums generated greater feelings of anxiety than posting to a CCK08 blog
Relationships
-
13 It was easier to form relationships with other course participants in the CCK08 forums than in the CCK08 blogs
-
14 There was a greater sense of closeness between CCK08 forum participants than between CCK08 bloggers
-
15 It was harder to form relationships that lasted beyond the end of the[J7] course in the CCK08 forums than it was in the CCK08 blogs
-
(Relationships/ connections between concepts in addition to those between people. Expected outcome M = moodler, B = blogger)
- 17 I find it more convenient in forums that all the relevant aspects and ideas are collected in one place. (M)
- 18 It's not very important for me that in forums, all the relevant aspects and ideas are collected in one place. (B)
- 19 I find it more convenient in forums that all the recently discussed aspects and ideas are quickly accessible. (M)
- 20 It's not very important for me that in forums, all the recently discussed aspects and ideas are quickly accessible. (B)
-
21I am excited when people bring lots of obviously related aspects to the table. (M)
-
22 I don't like musings about far-fetched associations and wonderings if they might be connected to the topic at hand. (M)
-
23 I don't want to be distracted by ever more links. (M)
-
24 I am fascinated that forums discussions rapidly connect to more aspects while the previous aspects are still lively present in mind. (M)
-
25 When new aspects and ideas come up, I like to explore their source and author, and sometimes they resonate with me when I learn more about their background. (B)
I like these statements - they get at the learning process. Whilst doing this I am constantly referring to my book (Doing your research project . A Guide for first time researchers in Education and Social Science - by Judith Bell) which I used during my Masters degree. Under the chapter on questionnaires it warns against the following:
Ambiguity, imprecision and assumptions in question wording
Asking questions which rely on memory (which is unreliable!)
Questions which ask for informatin that the respondents may not know or have readily to hand
Double questions e.g. Do you attend maths and chemistry classes?
Leading questions, e.g. Do you agree that mature students should have the right to express their views in tutorials
Presuming questions, e.g. Does the university make adequate provision for counselling
Hypothetical questions, e.g. If you had no family responsibilities and plenty of money, what would you do with your life
Offensive questions or questions that cover sensitive issues
I think we have to beware of all these! Very difficult!
I agree Jenny with your points. What a coincidence! I bought this book on Doing Your Research Project long time ago (the reprinted 1989 version). I used that for my researches, especially in education. I will re-read the book. I will review our questions too! I think this will take some more time. John 5/3/09
Reflection
- 27 The CCK08 forum discussions were less reflective than the CCK08 blog discussions
- 28 The CCK08 forum discussions were so fast that they prohibited reflection
- 29 There was less time for reflection in CCK08 forum discussions than in the CCK08 blogs
Differing perspectives
31 I have found these statements difficult to complete so have posted as far as I have got. If you think this is workth pursuing, then I will carry on another day. It's very difficult to come up with statements that will give us valuable information. You both might be more successful.
Autonomy
33 The CCK08 discussion forums were more teacher-controlled than the CCK08 blogs
34 There are fewer opportunities for self- expression
Dialogue
Still to do
Audience
36 Still to do
Cognitive style
36 Still to do(42ff are located after 151.- 187.)
My original hypotheses are:
1. Activists are mainly bloggers (primary) Moodlers (secondary)
2. Reflectors are mainly bloggers (primary) and some Moodlers (secondary)
3. Theorists are mainly bloggers (primary) and occasional Moodlers (secondary) (can be applicable to our Instructors - George Siemens and Stephen Downes, whom I think are theorists, don't you agree?)
4. Pragmatists could be mainly Moodlers (primary) and some bloggers (secondary) (and even lurkers - including those who left due to not meeting their interests, social needs, have found the course too theoretical, too complicated - not at the right level, or not having the time to do it etc.)
Based on my analysis of some of the forum and blog posts (subjective ones at this stage)
Hypotheses statements:
1. Activists, reflectors and theorists prefer to adopt blogging over Moodle forum discussion,
2. Reflectors and theorists would like to participate in Moodle forum discussion occasionally
3. Pragmatists may adopt both Moodle forum discussion and blogging
4. Most Pragmatists would prefer forum discussion over blogging
For 4. at least at the start of the course - with possible reasons of (a) easier to participate in discussion than writing a blog post, (b) easier to connect with others having a practical experience, (c) wanting to put new knowledge to use, (d) limited knowledge in the field or domain, so easier to participate in discussion in order to learn, (e) can try out strategies and ideas quickly - through learning and practice, and (f) can learn about practical situations in forum discussion.
Activists statements (Bloggers mainly) (primary)
151. I like to become involved in experiencing situations
152. I like new opportunities in learning
152. I like to maintain high profiles
153. I learn best when there is an element of risk in learning
154. I learn best when I could 'bounce off' and become energised by others (I would suggest that this is common in Blogger and Moodler)
Reflectors statements (Bloggers (primary) and Rare Moodlers (secondary)
161. I like to think things through
162. I like to listen, watch and gather data before committing myself to a decision or conclusion (mostly Blogger, could also be Moodler, if it is about discussion in a forum)
163. I tend to be cautious and conservative in the face of new knowledge and experience (mostly Blogger, could be first time Moodler))
164. I learn best when given time to think or mull things over (mostly Blogger, could be Moodler in discussion forum)
165. I learn best when I am given access to all available information to guide my decisions
166. I learn best when I am not pressured by deadlines or hasty decision-making processes
Theorists statements (Bloggers (primary) and occasional Moodlers (secondary)
171. I like to assimilate and synthesise new information
172. I like to fit new information into theories and explanation of how the world works
173. I learn best when I can use models and systems that make sense to me
174. I learn best when I can understand links and connections between facts and ideas
175. I learn best when I can explore links and connections between facts and ideas
176. I learn best when the subject matter is objective
177. I learn best when the subject matter is based on rational principles
Pragmatists statements (Mainly Moodlers (primary) and some Bloggers (secondary) (educators, instruction designers, education practitioners))
181. I am interested in ideas to see if they work
182. I think abstract ideas have little meaning for me
183. I like to solve practical problems
184. I want to put new knowledge to use (Bloggers and Moodlers)
185. I learn best when I can link ideas to real life situations (Bloggers, Moodlers and Lurkers)
186. I learn best when I can try out strategies (Bloggers)
187. I learn best when I can try out ideas
188. I learn best when I can deal with practical situations
By John 15/3/2009
Notes: The above statements could be ranked as SD to SA (Strongly Disagree -1 to Strongly Agree -5). When analysing the statements a score could be used to calculate (a) the mean score, (b) standard deviation to show if there is a correlation between the style of learning versus Bloggers/Moodlers and test the validity of the hypotheses statements.
There might be other or alternative categories under which we could organise statements like this.
Whilst working on these statements, I wondered if it might be preferrable to make a long list of statements and ask respondents to select which ones apply to them. We could either separate these statements into 'Moodlers' and 'bloggers' as shown below, or we could just lump them all together and ask people to state there preference, i.e. for forums or blogs on the CCK08 course at the end of the list.
I preferred blogging to Moodle forum discussions on the CCK08 course because…. (please select the response that you agree with)
42 I was already familiar with blogging
43 The blogging community was more friendly
44 There was more mutual respect between CCK08 bloggers than in the Moodle forums
45 I was less likely to be criticised on my blog
46 I could avoid the ‘loud forum voices’
47 I felt less anxious when blogging than when posting to the Moodle forums
48 I was able to forge closer relationships in the blogging community
49 I did not feel connected to the Moodle forum participants
50 Blogging allowed me more control over my own learning
51 I had more control over the topics I wanted to explore.
52 I could work at my own pace
53 Blogging allowed me time for reflection
54 I liked the ‘slowness’ of blogging
55 I was able to able to be truer to my own thinking in my blog posts
56 I had more opportunities for self-expression in my blog
57 Blogging allowed me to have a voice on the CCK08 course
58 Writing for my blog gave me more pleasure than writing in the Moodle discussion forums
69 Blogging allowed me more personal space
60 Blogging allowed me had more ownership over my learning environment
61 Blogging allowed me more freedom in the choice of discussion topics to follow
62 Blogging allowed me more freedom in the choice of people to follow
63 I found it easier to understand the CCK08 blog posts than the Moodle forum posts
64 I didn’t feel comfortable in the Moodle forums
65 I didn’t feel I had anything to contribute to the Moodle forum discussions
66 I could not make my voice heard in the Moodle forum discussions
67 The CCk08 bloggers often provided summaries of the weekly topics and this helped my learning
If you have any additional comments you would like to make please type them in the box below.
I preferred communicating in the Moodle forums to blogging on the CCK08 course because…. (please select the response that you agree with)
69 I was already familiar with posting to discussion forums
70 The Moodle forum discussions were more accessible
71 Blogging requires more effort than I wanted to make
72 I enjoyed the fast pace of the Moodle forums
73 I enjoyed the challenging discussion that took place in the Moodle forums
74 There was more lively debate in the Moodle forums
75 I was able to connect with a greater number of people in the Moodle forums
76 I felt more of a sense of community in the Moodle forums
77 I had a greater number of readers in the Moodle forums than I would have had on a blog
78 I preferred the linear style of the Moodle forum discussions
79 The discussion in the forums was more substantive than in the blogs
80 There was more knowledge creation in the Moodle forums
81 I received more response to my posts in the Moodle forums
82 The quality of posts in the Moodle forums was higher than on the blogs
83 The discussion in the Moodle forums was more flexible
84 It was easier to follow the discussion threads in the Moodle forums
85 It was easier to track back and locate information in the Moodle forums
86 There were fewer personal posts in the Moodle forums
87 I felt more comfortable in the Moodle forums
88 I was able to establish a presence in the Moodle forums
89 I felt I had a voice in the Moodle forums
90 I was able to make a greater contributions to the Moodle forums
If you have any additional comments you would like to make please type them in the box below.
People choose to blog in preference to posting to discussion forums because
92 They are more personal
93 They are more distributed
94 (=33) They are less teacher-centric
95 (=46) ‘Loud’ voices who drown everyone out can be more easily side-stepped
96 (=45) There is less criticism
97 There are fewer disparaging comments
98 (cf 32) There is a greater sense of personal control
99 (=6) There is a greater sense of belonging to a community
100 (=53) There is more time for reflection
101 (=56) There are greater opportunities for self-expression
102 There is greater opportunity for self-assessment
103 (cf 48) Relationships are closer and deeper
104 There is more crafting of writing
105 (cf 44) There are higher levels of mutual trust and respect
106 There is less posturing and pontification
107 There is a greater sense of freedom
People choose to discuss in forums in preference to blogging because
109 (cf 69) They are familiar – the easy option
110 (cf 72) Communication is faster/rapid fire
111 (cf 76) People feel a greater sense of proximity to other forum posters
112 (=71) Forum posting is less effort than blogging
113 (=78) Forum posting is more linear than blogging
114 (=88?) There is a greater community presence in forums
115 (cf 83) Conversation is more flexible/diverse
116 There is more sparring/challenge
117 (=82) The quality of posts is higher
118 (cf 4) They are more like a conversation
119 (=85) You can find information more easily
120 (=81) You get more feedback
121 (cf 75) It is a more efficient way of making connections
122 (cf 86) There is less ‘navel gazing’
123 (cf 121) You are more likely to receive feedback on your opinions
124 It takes too long to get a blog up and running
(How do people like the "big picture" ?)
131. I like it when the author enumerates many aspects that guide me around the big picture in his mind. (M)
132. I don't want to be urged to follow the author through his linear assembly of digressions. (B)
133. I don't like to be distracted to off-topics scattered across various side branches. I need a "red thread". (M)
134. I like links leading to further aspects if they promise a rewarding digression when I actually click and explore them. (B)
141.(a)-(c), 142. (John's questions about Connection strengths) see Questionnaire Draft 5
(151. - 187. above between 34. and 42.)
Comments (10)
Jenny Mackness said
at 5:18 am on Mar 1, 2009
Sorry again about the formatting. I'm too tired to sort it out now - but will come back to it some time. This is just a brainstorm for a deeper level to the questionnaire. I haven't had time yet to take in your ideas about cognitive styles Matthias. I also think that I might be off at a tangent from both your thinking here, so please do both say if this is not what you had in mind. I was wondering though Matthias whether the comments you sent in your email about the fuzzy notions around forums and blogs differences, might be the kind of writing we would include in a discussion when we write the paper.
John I know you made some comments about the questionnaire, which I now can't find - which is a very good reason, I think, for keeping any substantive discussion about the research here on this wiki. I'm sure it's there somewhere in my email list, but for now I've lost it. Sorry. I haven't ignored you!
suifaijohnmak said
at 5:32 pm on Mar 1, 2009
Hi Jenny,
That would be great.
I have a quick look at the questionnaire. A fantastic piece of work from you. Just excellent.
The only minor amendment in the form for the Date to be changed from December 2009 to 2008 (the last paragraph).
How about the following questions on:
1. What are your purposes of blogging (in CCK08)? (either have this open or a few choices if that is appropriate). The respondents could check on a number of choices.
2. How valuable was blogging to you (in CCK08)? Highly valuable.....Not valuable
3. How important was blogging to you (in CCK08)? Highly important.... Not important
Reasons for these question are - there may be others who don't want to proceed with the Skype interview and we could still know why they blog, the value and importance of blogging in the first place.
I have written a post on blogging where I list the major purposes of blogging that may be used for your reference.
John
suifaijohnmak said
at 11:45 pm on Mar 1, 2009
Would a concept map on those 6 aspects/statements help? I think we could visualise the relationships with such a map. It would take some time to prepare such a map. But we (and I) could try. What do you think?
John
x28de said
at 11:56 pm on Mar 1, 2009
I think we could perhaps obtain help with the questions by recollecting the main connectivist ideas, which cover both connections among PEOPLE and connections among concepts, and connections of varying and growing strengths. Our hypotheses seem to be that these connections are DIFFERENT with moodlers and bloggers, and probably also the respective weight of people vs. concepts/ topics. (In my email of Feb 26 filed under Research References\CCK08 reasons for blogging quotes.doc, I disussed people's connections under 1., concept connections under 2., and their respective weight under 3., as I am noticing only now.)
The difficult part is HOW the pereferred types of connections differ. My guess is that moodlers are more CONvergent thinkers who love to bring numerous topics to the table that are strongly tied and obviously related, and love to have all the aspects in one spatio-temporal place. Bloggers, OTOH, are IMO DIVergent thinkers, who like to relate to more distantly related, distributed ideas, and the conceptual connections are initially weak getting stronger by reflection.
x28de said
at 12:06 am on Mar 2, 2009
John (I was already typing when your comment appeared), yes, a concept map would be great. It could also reveal where the 6 aspects originate from a slightly different background and are related to learning SUCCESS, not necessaritly to incremental
or latent learning, and more about pleasure of the result than feeling comfortable during the process. In particular, "emotional significance" seems to be applied slightly different: to the learning result rather than the people during the process.
Jenny Mackness said
at 10:47 pm on Mar 2, 2009
Wow - what a productive weekend! Thanks to you both for your comments and ideas.
John - thanks for finding the email with your comments for the questionnaire. I'll have another look at the first 3 parts to the questionnaire later this week.
John - Like Matthias I think developing a concept map would be a great idea and potentially a very usefull addition to a paper. You and Matthias are both good at these. I look forward to learning from you both.
Matthias - I think we are both thinking along the same lines about how to organise a Part 4 to the questionnaire. As I was working on this on Saturday - it did occur to me that we should be grouping questions around connectivism ideas. Perhaps grouped under Autonomy, Diversity, Openess and Connecteness? This would be recognisable to respondents. How would these categories fit with the thinking you laid out in your email of Feb 26th Matthias?. I'm going to create a new page and call it Part 4 Draft 2 - where we can try and cluster our likert style type statements under these headings (or alternative headings, if you think others are more appropriate.)
Jenny Mackness said
at 11:04 pm on Mar 2, 2009
I don't know who formatted the page - but a big thank you!
suifaijohnmak said
at 8:51 am on Mar 11, 2009
Use of jargons in questions and discussion
I found myself using a lot of "jargons" and sound more like a "scholar" when crafting my views and entering into discussion. I have also noted that the intensive use of unfamiliar words being included in our questionnaire. I don't think we have intended in doing so. But, when I entered into the "scholar" mode of communication, I found myself gradually fall in love with words of beauty (jargons). And I think it is OK.
Use of simple words or phrase in questions
I am putting myself into the respondents' position now. I think there are some respondents who may not have a full understanding of the terms like technology affordance, enumerates, sparring (do you mean sparing?), linear (this is a concept), (as mentioned by Jenny) may I suggest that we replace those words with simple ones? Otherwise, we have include a definition of terms in the questionnaire or interview (to prevent misunderstanding).
Questions format
Re-phrase some leading questions - I like....if they.... (phrase them into 2 questions - with one I like..... (if they)... I like.... and check the if condition aligns to the first queston if possible. So, delete the conditional parts if necessary. Example: "I like links leading to further aspects if they promise a rewarding digression when I actually click and explore them". The further aspects is a vague one. So it could be reduced to one statement: I like links leading me to explore new knowledge or information. We could include the degree of importance for each of the statements, or the interviewer may need to check the degree of importance and satisfaction in each statement followed by how and why elaborations.
suifaijohnmak said
at 8:59 am on Mar 11, 2009
When I went throught the questions, I noted that all questions have asked for a response based on personal preference. Such responses are sometimes based on a combination of knowledge, experiences and emotions (feelings) of the respondents. These responses could be changing (refer to Complexity and Chaos Theory) as a result of connections, interactions and learning. As our measurement here is AFTER the course (IN RETROSPECTIVE of the preferences and feelings at that time), we may need to ensure that respondents are reminded to state their preferences based on what they feel DURING AND AT THE END OF THE COURSE instead of NOW, or else it would be confusing in the analysis. I state this to highlight the importance of the principles of Chaos and Complexity Theory, which indicated that any changes in the initial conditions would have a significant impact on the changes that follow. So, I have been thinking about any related principles of those Theories that could be incorporated into the questions - to illustrate the network dynamics. The "Catherine", "trolling", "power", "debates between groups and networks", and the "shifts in learning foci" sort of effect. These development may be perceived by many to have dramatic changes in the "balance of attractions/attractors". This results in emergent learning which NO ONE has ever anticipated. This has taught me (may be us) how to act and react in extreme situations in e-learning and network learning at a social level. These were based on my limited perception. But each participant may have views different from mine, though some have raised them via the forum or blogs but many others might not have raised them as yet. These may be revealed more thoroughly through this survey research.
Would you like to review the questions in light of my comments above? In principle: keep them simple, no leading question, and if it is too complicated, either rephrase it or break it into two or three questions. How does it sound?
John
x28de said
at 10:23 am on Mar 11, 2009
John, I fully agreed with your statements about language, but somehow my comment went to the frontpage instead of here.
You don't have permission to comment on this page.