Assessment One - last post (I hope)

Here I have taken the marking criteria and attempted to match to what I have done over the semester.

I don't think I have a good job of this but I must remember my trepidation at doing this course. I had explained to Oriel that I would only do one this semester as Evaluation would play 'second fiddle' to any other.

Picture to show feelings

(Not many can say they have an inflatable Scream to provide a picture)

Investigate evaluation theory and processes in your context - Okay I know about ADDIE and think that is theory I definitely saw as something I could understand. A needs evaluation fitted my situation but I will be interested to see how the other forms of evaluation worked for others as I didn’t get a chance to really get a grip with what other were doing especially in the case of Jon and Kevin’s. I also would be interested to see how they relate to Bronwyn’s comments about how things can be done differently next year. They fact that Jon had to take the lead so much from Kevin (the situation dictated it) and also the issue they had with contacting past students. I felt they had a lot more bridges to cross than maybe other groups.

I found the whole evaluation theory interesting in relation to my work place as I don’t think it is always something that is done to any great level with just maybe just the surveys done and results not always looked at (I came across an example at school while doing my report). Presenting the final report won’t happen until next year and again in what form it will take I will not know.

· Demonstrate evidence of reflective learning about the process of evaluation - this was an area I felt I worked at a lot at as evaluation in this context was new to me. Normally I would look at my lessons, children's learning but not in a more holistic wider approach as required by this course and the assessment that we needed to complete.

In one blog post as I was deciding on what was to be the topic I looked at how 'tools vs. purpose'. I looked at the school situation, talked to colleagues, talked to others in a similar situation and also looked at documentation that would help my understanding further. This also allowed me to change my original ideas. I still feel further work will be needed in the future about tools but purpose and in this case professional development played a greater part.

I have looked at eLearning guidelines; related them and used them within my report.

* Reports and reflects on the progress of the evaluation project – I have regularly given updates on how my project is going and also what steps were to be undertaken next. The language and pictures used were aimed at lighten the mood.

* Critiques at least two other evaluation projects of participants in the course.

I worked constantly with another course member to construct our joint report. In a way this was a critique as it involved looking at another's work along side your own.

I regularly commented on others work both on the discussion page when the projects were being created and within the blog: sometimes the purpose was to communicate about issues you were having. Other times it was to reflect on what they had written.

On the completion of reports I have commented on others’ work.

* Discusses and critiques the findings of their evaluation project – I did this briefly on the discussion page on wikieducator and also in my second to last blog post. This I did not feel I have done as well as time I felt did not allow for a full and frank discussion as such reports should be given.

Well everyone have a good break and I hope to meet some of you next semester or in the real world. Thanks Bronwyn for you vampire skills. I need to acknowledge my husband support here - proof reading and playing solo dad on a number of occasions. I will leave you with a picture of our shared desk which will see became multiscreened (most I have had is 3) something I highly recommend.

Final report - enjoy!

Here is my final report.

Finished it last week but was having problems publishing in Google Docs (oh it was such a simple problem - be in edit mode!)

A Quick Summary of report:
A needs assessment was conducted with a primary school’s teaching staff at the conclusion of a three year ICT cluster contract. This type of evaluation was chosen to investigate what professional development needs to be put in place for teachers to continue to teach ICT effectively.

Recommendations:

• Improve sustainability practises so all staff have a point of reference to refer to that is well documented.
• Any professional development to be in line with the level taught, short in form, opportunities to revisit and non-negotiable.
• Collaborative learning needs to be formalised across the school.
• Teachers to be given opportunities to learn within a group, given formal input (from inside and outside the school), opportunities for directed sandpit time and further support both in the form of colleagues and/or documentation.

To use a language all would understand - good practices are being lost and lacked direction. I plan to publish my report for the staff first through management. This I feel would be the right protocol even though I am doing this course/report outside my position.

Some of the recommendations will be looked into and addressed through my 'Special Topic' to be completed next semester where I plan to create a wiki/website to support staff. I will use some of their recommendations as to how they learn to help with this.

Identify and rationalise eLearning guidelines

I must have missed this bit. I definitely read the work on it as I have used two of them for my assignment so I will rationalise them here as to why I used them. (actually just found a comment from Bronwyn on this in one of my blog posts - Oops!)

Identify and rationalise two eLearning guidelines which can be used to measure quality in your context.

TD11 Should staff use a team approach to develop the ICT skills they require, and to teach ICT applications confidently?

TO10 Is good eLearning practice available to staff in a way they can adopt and adapt in their own work?

I found the guideline layout with the cube interesting and spent a lot of time clicking the different options. When looking at these two within the context of my situation they were chosen as they allowed me (wearing my evaluator’s hat) to look at staff and best practise and then relate them to ICT (Information Communication Technology) which I teach part-time to some of the school. (as a footnote in my survey this came up about only part of the school having someone teaching ICT)

To explain further in my present situation as a part time teacher within a school that previously I was a senior teacher I have previously wore a number of hats but at present are more of an observer. I have a feeling that that ICT contract has not always worked as well for some when it came to ICT as other element took over (see the final report for more on this – will provide the virtual popcorn). I saw the team work and best practise definitely fitted the school’s need as not all staff has had the same experience over the three years – they had voiced this to me informally.

Interestingly as I have worked through my project from the plan to report stage collaboration and sustainability two areas also used within the guidelines featured in my work. So on reflection and revisiting the guidelines others could also have been used.

Not final post

Just spent ages doing Assignment One when I good be enjoying finishing my report and it wont post to my blog for some reason so will try and do in my lunch hour tomorrow!

Rather annoyed - have to learn how to swear

Pressing the Flesh

Love that saying 'Pressing the Flesh'! Used it to describe that I had the pleasure of meeting a number of our class in person. This was at MIT for a working lunch! Lovely food Vicki! Present were Oriel, Dana (he has 11 cats!), Mareena, Misha and Me and the guest of honor Bronwyn who we made work for her money by seeing her afterwards about our assignments. Nice to meet those who I feel I know.

I have made some general notes which I will write up and link in my next blog post once I have made sense of them.


By the way those involved in the discussion the other night that the plan should be 50% and the report 25% Bronwyn agrees. (Doesn't mean it will change now)

The reason for the potato picture is we are all getting to the stage that the fruits of our labour will be showing - ie survey results coming in. These are the fruits of my labour involving three grow bags. Harvested now so I can get the tomatoes in.

The Famous Five Met

A small group of us still met online tonight - Dana, Lousie, Liz (from very hot Perth), Rachel and Me. Allowing for Bronwyn apologising before hand that she may not make it we still spent an hour plus looking at issues we have had, any help we can give (Rachel tried her hardest - thanks!) and also looked at our special topic (yes the next course). Dana has nearly completed his and the rest are due to do ours next year. So we looked at what we thought we would do. We even looked at the fact that the plan is 25% of the course but the sweat and blood involved was more like 50% so maybe Bronwyn should swap the marks around with the final report.


Is it time to revisit my previous photo just to remind you of how we felt.


Data collection - I now have 11 completed surveys and sent out the survey to 16. Some teachers had issues with doing the survey online. I think user error is involved so I have given those that contacted me a paper copy.

Also I have completed two interviews using Audacity so I could record and listen to it later. Each interview has taken around 20 minutes. I hope to interview two more teachers and ideally one more who is a lead teacher and a team leader. I feel her interview will bring a balance between the two 'sides' I have talked two so far.

Next point: the rest of the interviews and surveys, looking at the exemplars provided and think how I will structure my final report.

Survey started

Lisa and I have both released our survey out onto the staff. So far between us we have 16 results in. I marked my email so I could see who have read it. (10 read emails and 7 completed surveys)

One teacher was unsure about privacy which I had mentioned verbally so have sent out an additional email addressing this issue and mentioned that they can see me if wishing to know more. With Obsurvey we only know the city location which means nothing in this case.

Having a few issues timetabling my interviews as I have had two shorter weeks (and I work part-time) because of PD and Union meeting so will have to try and fit them in next week. Have set up Audacity as my recording device on my laptop and will save them as MP3s.

Hope all one has had a good long weekend.

Now it can be marked...

In her attempt to get the last drop of blood from Lisa and Katie Bronwyn sends out her feedback J

Sent this line to Bronwyn in a recent email as Lisa and I continue to polish. Now it is time for marking.

Have sent out my survey but it got blocked by the firewall at work so hope the AP who is acting principal see my email and unlocks so gathering can now start.

Next step is to comment on some of your projects and to check out Audacity which I am going to use as a way of recording my interviews.

Have not put a copyright for the picture - so all of you may use it if you wish.

Survey

As Lisa and I have got into the habit on a Wednesday night we had our usual assignment chat. The focus this time was our online survey. We are using Obsurvey as mentioned in a previous post. I 'found' this through another site and it seems to have advantages over Survey Monkey.

Even though this meeting could of been a short one we spent time a lot of time doing a mock up of it to check if there are any issues. We are doing our surveys individually so we can compare them in our report if anything comes to light in the survey.
To use an old saying 'the proof is in the pudding'.

I love rich text editor

Here is an update of where the swimming pool was. My poor brother-in-law is shocked that the deck (can't fit in photo what is round the side of the house) is bigger than his flat.

Please note the title of my blog this is my new swear word. For those of you who have not used rich text editor much will understand why this code is treated with such distain. (Foot note I tried to have the code for page break as my title and the blog couldn't cope - typical!)

Lisa was away over the holidays entertaining any child that would call her aunty. I was seeing distant family and having a builder in everyday ...and I kept the cake tin full! Because we were unable to make contact during this time I was involved in a 4 hour session of corrections suggested by Bronwyn.

To quote Bronwyn (sorry not APA referenced) 'Your evaluation plan is looking very good and you are not far from completing it.' I tried to use this as my mantra.

Our evaluation is at the point where if we play with it anymore bits will drop off. We have looked into using another online survey rather than SurveyMonkey which has its limitations. You will see a link in my blog (two blogs ago) and in Lisa's.

Plan Here is a link to our plan. Make comments here or on the discussion tab on our project.

Making changes

Thanks Bronwyn for the feedback. Have yet to hear from Lisa so have done some of the changes but will need to get out the legendary big piece of paper to make sense how we could adjust the questions. Husband away so may not get a chance until tomorrow or Monday night to start.

Funny experience with last blogpost. Got three comments from outside which included two comments from the person who developed the online survey I was thinking of using. He is in Denmark. When I get a chance will ask a couple of questions I have about his work. Nice to see what motivated him to develop it.

Survey Monkey or ???

As I wait for new printer cartridges so I can print out the edits Bronwyn has made and look at it off screen I have been looking at other survey tools.

Survey Monkey is good until you want to look at results - the choice is paying or doing lots of screen shots.

I tried Rachel's suggestions on Kevin and Jon's wikieducator page - zoomerang and limesurvey (both weren't right - wanted too much personal information.) One site I have used before had the most amazing list - finally I found Obsurvey (Have not used it before!)

So I have created a survey. Was planning to get you fill it in but having a problem so watch this space.

Tools vs Purpose

Originally Lisa and I were planning answer 'What resources are needed for teachers to teach ICT effectively? but on reflection the word resources changed to professional development.

Why was this decided?

Which comes first? Clearly professional development needs to be evaluated first rather than resources.
'a heightened awareness that the need for professional development in the area was likely to continue after the end of the project' Taken from a Ministry of Education report. The project mentioned is the ICT cluster project that both Lisa and my school are due to finish.

Professional development takes a number of forms and what we want to do is find out what works for staff rather than assume you know. Coming to the end of this contract is an ideal time to look at what a school could do for its staff rather than get 'bogged down' with tools.

Before starting the report I spoke to a number of people and one view stood out. Should the next step for the schools be 'tool-based' or 'purpose-based'. Often what happens is a new tool gets people excited but serves no purpose. An example of this was a teacher told me how her school wanted Skype but no one uses it. A recent reading '100 ways Google can make you a better educator' listed all these lovely tools but ordered them under a purpose heading eg collaboration, news, sharing. This is the direction I would go after looking at professional development.

Questions - getting them right

I just tried to some work on quality but having problems downloading so will look at questions - the questions you ask to get an understanding of the situation.

These are a set of questions that I found in the chapter Bronwyn send us and I have tried to keep these in mind.

Reeves, Thomas, C. & John G. Hedberg (2003). ''Interactive Learning Systems Evaluation.'' Englewood Cliffs: Educational Publications.

The questions are not always easy but I was interested that one course colleague mentioned that if she used 'Survey Monkey' it only allowed 10 questions when using the free option. I quickly checked how many I had and then thought 'Why would you want to ask more?'.

I have just spend ages trying to find a quote I liked but will have to just guess - it basically said that the questions are not set in stone and must allow for the direction the interview goes. I also liked what Reeves said when you gather this information 'obtain ranking of information from each person for whom you speak'.

And I thought we were done....

Lisa had worked on the finished date of the 19th (yesterday). We had got to the point where one question for our interview kept us busy. We thought we were finished.

BUT now we have to add some bits on quality which I think I won't feel like doing until Thursday. I have added this photo of the mess outside my back door where my husband has removed our swimming pool (getting old), deck and some of the fence. We don't need a storm for this to happen. When I look at it I think of the stages of an evaluation plan - background, decisions and of course the final result.

Please note the new swimming pool forming at the bottom.
If you get a chance have a look at our plan.

We will be in on the 19th

Lisa and I decided to have a phone marathon and keep going with our report. We have had issues with method and with the help of Bronwyn we have been introduced to a data-matrix. Because we wanted to check that our sub-questions can be answered we are working on the survey questions. Not sure if this is the order of things so have asked on the discussion group.

Have kept an eye on what others have been doing and get the feeling that some are 'keeping their cards close to their chest' and will reveal all when ready.

Here we go - we are adding to the wiki

Lisa and I have managed to have a phone conversation (the advantage of living in the same city). After 'chewing the cud' about our respective work places we worked through what we had both done so far on the wiki and which questions need work on. The plan is that both of us will add bits with our initials next to it. Then on Tuesday as I have the day off will mold together the plan and we will make contact again to check it is what we want then let the rest of you add your feedback.

Preparation, preparation, preparation ...

I feel like someone in waiting as Lisa is going through ERO this week so in the nicest way I say 'not worth knowing'. I have shared with Lisa what I am looking at but have yet to hear what she is doing so will get another large sheet of paper out and scribble some more under the headings.

Below is the summary Bronwyn put on the blog for me this week so I will reply to it in red.

'Katie has formulated an excellent big evaluation question. It appears she will not only need to find out what sort of technologies/tools the teachers would like to use Some have already expressed their wishes but not sure how much thought have gone into them. Eg they want Skype but not sure why. To type or to speak which will also bring up the issue with sound cards not being in all computers.
, but also what they will need to use to keep up with educational trends this is an area of concern - sustainability will be an issue if certain key staff leave (I am at end of year - shhh!!),
and maybe later she can find out more about what is going to be relevant for students to use (getting staff to turn on computers and let the children use them is a major issue). I have suggested that a good starting point might be Derek Wenmouth's post - 100 Ways Google Can Make You a Better Educator. Some of you may know him - he critiques the latest strategies and technologies for the school sector. (Have looked at Derek's list and think that if I break it down so that teacher have to pick a limited list that they want to use) Katie has made a great start with the thinking for her part of the evaluation plan. Where will she head next?

Where will I head next? Yes large piece of paper would say a glass of wine but I behave myself during the week. Plan to look at Kevin's questions that he listed in a discussion and see how they will help my train of thought.
Kia Kaha people (Keep Strong)

Very proud of my large sheet of paper

In my last blog I said I was going to get out a large piece of paper. I did and I wish that could be my report. I took a few of the headings and added some notes then drew lots of arrows between. I have the big question and have come up with some of the sub-questions. Feel good with what I got done.I think I need to look at who is involved.
As Lisa and I have just become partners I need to slow down a bit and get her up to speed. I also need to fully understand what she wishes to do. Have emailed her about this today.
Have looked at the education guidelines and picked two that I think will help but will wait until I have worked further with Lisa then post then in their own blog post. Also looked at the 'cookbook' and took comfort from the fact I think I am on the right track - maybe I should wait until Bronwyn looks at it!

Week Six - Planning Evaluation Report

Was feeling a bit unsure about what I wanted to do as my Evaluation topic. After an announcement made in a staff meeting about some new computer work to be done I came up with a topic. Actually emailed the senior management team at midnight. To explain -

As a school we are coming to the end of an ICT contract of 3 years. I have been rarely involved as I am part-time. Most of the focus has been on mind-mapping (SOLO) the rest involving the ICT I am lead to believe has not been as successful as a lot of the information has not been passed on to those other than the lead teachers.

The school is one of the first in Auckland involved in SNUP where schools are getting optic fibre put in and in our case a new server. This will bring a lot of resource issue up. A lot of classes have not got enough plugs or network points for their computers. Recently they (the teachers) were asked to fill out a simple survey (created by someone off site) as to which applications they can use, want more help or need support.


I have got the survey and on reading it it shows that teachers were either not shown or shown too many applications some with no relevance to them. As one teacher said she wanted professional development 'appropriate to my needs and class level'. I also heard through another teacher at another school 'why not teach 6 applications and do them well'.

This is the point I am. To find 6 applications (or less) that each level of the school can use in their class. I may focus on just the top end (middle and seniors) as these are the classes I work in.
Tonight I am getting out a large piece of paper and start scribbling some ideas - find this a good way to work definitely need to look in the Evaluation Cookbook.

My big question
'What resources need to be put into place for teachers to teach ICT effectively?'

OIL module - Digital Learning

Hi here is posting for OIL module. It was completed when we were a three so have left Alex's comments here as we have used a comparison form rather than a blended comment.

My Evalution of so far - Week 3

Having just read Bronwyn's posting about our blogs so far I thought I would do the same. I think I am in a unique situation in that I don't teach a course so are thinking long and hard about what I should evaluate. I have some opinions but to make the whole process triangular I have problems. I am doing my project outside by workplace but about them...(time pause -had a shower and coffee must of helped the brain) another option is to interview the children and the teachers using a needs analysis.

Have got work done on the case study but waiting for partner contact which is hard as I (like others) have limited course work time.

Evaluation has hit me hard to the point that I include it into my life. Recently we got a quote for insulation. We decided not to do it but my husband commented on why the company didn't make any further contact with us - should this be evaluated and included in part of their company's practice. evaluation on the brain - not good!
I am not sure how to record what I have read - has anyone got a way that works for them and want to share.

Cartoon

Why evaluate?
Was having a little search around and found this article.
But I actually like this cartoon.

Taster - Evaluation

Taster
Rachel and I have been busy 'working behind the scenes' establishing our partnership and drafting the evaluation of our Taster case study. We have now finished so have a read and if you have any comments please comment on my blog so Rachel and I see how we are going.

Draft - Evaluation and quality

• Why is evaluation important to you and how do you define it?
In the beginning of my long path as a primary teacher (over 20 years) we were always taught that evaluation was the last part of a lesson where you were to evaluate both yourself and the child and this was to dictate the form the next lesson was to take.
This in its simplest form is why evaluation is important.
How would I define evaluation? To use an analogy it is like doing back stitch. To move the stitch forward you must always go back first and revisit where you have just been...and when it is going well you check up less often!
• What sort of evaluations mentioned on the presentation is familiar to you already and why?
Summative and Formative evaluation is very familiar as teachers we do it a lot and with the introduction of the National Standards even more. Also this has ‘appeared’ in other courses.
• Why is quality important in eLearning?
As mentioned in the readings quality evaluation is important often the saying ‘blind leading the blind’ could be referred to many lessons I have seen.
You could look at value for money. Was the course etc expensive to create, resource and fund? Often this is where criticism from outsider will focus.
The eLearning Guidelines have ‘been designed for teachers, support staff and managers, and provide up-to-date, practical information and direction on e-learning.’ This I found an excellent tool when looking at quality. In line with my work place especially in terms of the teaching relationships four areas I found important: Sustainability (what would happen if staff leave – at present the knowledge has gone); Collaboration (sharing information – said, not recorded); Learner-centred (diversity of students – presently expectations not kept high); and Innovation (exploring new ways – does too much at once and badly).
In all those guidelines above quality is an issue.

Taster - Evaluation

Have had a look at the Tasters and have chosen the Engineering Mechanics as it requires a need-based evaluation which is what I will be using for my Evaluation Report.

I have posted it here as I would be interested in what others have done. Will send to Rachel for her input.

Triangulation - the inword

Jakob Nielsen's Alertbox for July looks at 'Interviewing Users' This is good reading to make you think about what part you want the interview to play in your research. Used the word 'triangulation' like some of the work that Bronwyn has presented.

Google Reader

Have taken Bronwyn's suggestion and used Google Reader to keep an eye on others' blogs. Have used NetVibes before as a reader so this will be a good use of Google Apps which I am just starting to investigate. Have now added it to my iGoogle page so I will be able to keep up.

Some other students do not have blogs that allow you to comment on so we have to see what they want us to do as the course progresses.

Two quotes from a study recently released

Have posted the link below:

... another study he conducted himself found that Britain's top private schools produced the best educational results with far less use of technology than state schools.

"If you put a fridge in a child's bedroom they will eat more," Dr Sigman said. "If you put a screen in their bedroom they are going to watch more."

Link

My Evaluation report - possible ideas

The school has been involved for the last two year in an ICT cluster contract. I have not been involved we part-timers are not always included. I was involved in one afternoon as a teacher was absent - waste of time!

The impression I get from a lot of the teachers is that they didn't get a lot from it. The lead teachers who were sent on training days were meant to share it with others. I have never received anything by email. I asked the principal about this and she said that it is done in team meetings (I am not at them). On further asking of staff they said they hadn't received much at all.

As it is near the end we were all given a survey to complete (interestingly on paper) - I have not heard what came as a result of this. I got to do it as well.

I thought I could to a needs-based report using some teacher (not lead-teachers) look at what they got from the training and what they wanted and maybe included the children to see what they have got from it. This is a bit harder.

This is not going to be easy as I was not involved in the training but this is the best I can think of on a Saturday night!!

 Added this ADDIE Model as I think it will become my mantra for this course.

What I do with my day!

Bronwyn asked a question in her comment about whether I teach a topic across the board the answer is yes and no. I work two days a week in a primary school releasing teacher for approximately half a day four times a week. One class I do every week for half a day. I try to take ICT. I use the word try strongly.

To explain. The school has 16 computers in a COW (computer on wheels) which I move around the school but only if it is not raining as you should see one of the hills I have to go up. I have nearly lost the whole set up going down. I have had issues with the wireless not working - it is not in all the school so have to use a 'nail dryer' as I call it but that will go missing!! I do ICT as sometimes I can't get on the Internet so have to be program based - did Powerpoint last term. eLearning is not happening!!

I refer to my school as technically challenged as it took me a whole term to get a computer so I could just use as a data projector. I must have been the only person teaching ICT who yelled out the instructions.

Right now back to comments about evaluation.

Introduction

• Who you are and your work etc., 

Hi my name is Katie Gibbs and I live in Auckland. Originally I come from Taranaki and taught in England for 13 years.

I am a primary school teacher who teaches part-time two days a week where I try to do ICT with a technically challenged school.

• Your interest in evaluation and what you know already, 

My interest ... to be honest last paper! what do I know - not a lot! As I don't teach a course I will find this challenging and I am that one that asks lots of questions. Sorry!!

• What you hope to get out of this course, 

After navigating my way round I hope to pass this course. Sorry I think I am still developing my thoughts here.

• Ideas you have for an evaluation project. 

 Are you able to do a project that evaluate what is not in place and suggest what could be - no eLearning in the building to evaluate.

Katie

First Day

Okay I read and tried to digest - Wikieducator does look different.
So welcome to my blog - if I get time I will add some widgets.

Katie