How could I evaluate a learning and teaching episode?

Background

It is the start of trimester and my timetable is jam packed full of great classes to teach. My academic studies coordinator approaches me and asks if I would like to take over the unit coordinating role of one of my classes, my instant reaction is ‘absolutely’, however I am not quite sure what it is going to involve and upon my discovery, the class materials, assessments and student feedback wasn’t what I was expecting for this unit.

The course is a Critical Studies unit titled ‘Jazz 1 – History and Theory’ it is delivered over 12 weeks, 10 of these weeks is delivery of content with week 11 dedicated to revision and week 12 as assessment week. The course is run at both Melbourne and Sydney campuses and comprises of a 4 hour lecture in total which is broken into 2 parts. Students within this class are trimester 4 – 6 students, the class is an elective which is offered every trimester and has between 10 – 25 students per semester, for many students this is the last academic studies class they will have to undertake in their Bachelor degree. As this unit is situated within the Academic Studies department, any student regardless of discipline (Musical Theatre, Contemporary performance, Classical performance and Composition) can participate in the class.

This post will provide a plan that I will use to evaluate this particular episode, and will provide links back to the previous post for best teaching practices within music education.

The Episode

When I received the unit materials, there were four issues:

1. There was no clear alignment of history and theory topics week to week;
2. The information for students on the lecture slides was vague with no media (audio/video) embedded…. It’s a music course right?!?
3. There is no practice exam for the theory content in week 11
4. There are no worksheets throughout the course to consolidate theory content/

Upon completing my rewrite of the course, I am in need to reflect and evaluate how the rewrite is perceived by students, and as well as other staff members who teach the course at the Sydney campus. I am curious whether the student grade average had increased or decreased, I would also like to find out if the students are satisfied with the following:

  1. Is the theory information relevant to the history topics and do the theory topics unpack in a logical way week to week.
  2. Are the additions of new historical figures ‘important’ enough to be discussing within the evolution of jazz?
  3. Is the theory too hard?
  4. Are the exams too long or too short? Are students completing the entire theory exam?
  5. Are the new handouts appropriate, are students completing them?
  6. Are the readings too long or too short?
  7. Are students reviewing lecture slides? Do the links work to the media?
  8. Do any of the topics and/or content double up with other subjects that students may have already done?
  9. Are the students happy with the delivery, content and expectations that this subject places on them.

The Rewrite

As we beginning the new teaching year I have approximately 3 weeks to review, map out and think about the assessment component for this course. I begin with the research essay topics, students have the option of choosing one of three topics to research and discuss, I find out what the questions where before and found that they weren’t too bad, so I construct my own essay topics and place this into the Extended Unit Outline (EUO).

Next, the theory exam, a review of this I notice that there were questions that were confusing and one question that had no relevance to this particular theory course with no direct link to the use of this theory in historical context, also there was also no type of analysis question, I revised the assessment however my next concern is that the exam is now too long for a 90 minute exam comprising of 140 marks.

Upon reviewing the history/listening exam I noticed the following errors:

  • Not enough room to write short answers
  • Identification of stylistic traits and analysis is too open ended

I developed the exam to include more target questions and for students to write in complete sentences (not dot points). Students are required now to listen to 8 songs, identify the track name, artist, genre and time period and discuss 2 points that are important to the development of jazz in terms of social/political, importance of the artist and stylistic traits.

Next part of the rewrite is the lecture content. Traditionally students will receive the lecture slides prior to the class to follow along with and then review in their own study time. However, the issue with the slides I received for both history and theory were that large chunks of text from the readings were photocopied and placed into the slides. The slides I revised now consist of key terms, summarised points of the readings with page number reference for students to investigate further and media examples (audio/video) with direct links to online content.

Now the course is revised, history and theory aligned and lecture content reworked, it is now time to deliver the course over 10 weeks.

Plan to Evaluate

Evaluation is an essential component within all disciplines, analysis of the planning, implementation of a lesson and the impacts of student learning can determine where teachers can focus their efforts in making revisions and which parts of instructional methods and materials will remain within their future lessons (Danielson, 2007).

My plan to evaluate is much likened to the musical sonata form of a symphony, it will involve an introduction by discussing what and how I am going to evaluate, the exposition which will involve the themes of my evaluation, the development will go through how I will obtain the feedback and data, the recapitulation – what will I do with the feedback and data and then finally the coda – implementation of the evaluation.

Introduction

The way this plan will unfold is within the student evaluation of teaching approach where students will provide feedback of which will lead to become a learning focused approach where learning activities and materials will be enhanced/modified or further developed in order to construct the students own knowledge (Tran, 2015)

The first part of the plan is to decide which sources of information to draw from and the methods of which I will collect the data (Harvey, 1998). Qualitative data will be used to assess student feedback surveys and previous trimester results, where as Quantitative data will be used to receive feedback via short answer questionnaires from students and colleagues.

Exposition

As mentioned previously in this post, there are 9 factors of which I wish to evaluate within the course; these can be broken down into 3 distinct themes of evaluation.

Teacher Perceptions Student Perceptions Student performance
– Relevance of theory to history

– Importance of new historical figures

– Topics double up with other subjects?

 

– Theory too hard?

– Readings too long or short?

– Students reviewing slides/access to media

– Student happiness with content etc.

– Completion of theory exam, too long or too short?

– Are students completing handouts?

 

By identifying these three themes I can then measure the student understanding/perceptions of the content throughout the course. This will also provide me with an understanding of student engagement within the lecture slides and exams. Using these three themes to understand perception and performance I can then develop the course further, based on Danielson’s four domain framework of planning and preparation, classroom environment, instruction and professional responsibilities (Danielson, 2007) to identify and enhance areas that require attention. (Stronge, 2005)

Development

It is now time to consider how I will obtain the data and which type of data I will use. Considering the right series of questions and methodology (Harvey, 1998) will be made easier by taking the three themes and categorizing further by modelling the Evidence Checklist for Teaching Awards (Department of Education and Training, 2016).

Peer

Self

Student Reactions

Student Learning

– Review of course material

– Review of assessment material

 

Self reflection on student feedback

– Critical reflection on content delivery

– Evaluation of content

– Evaluation of assessments and worksheets

– Completion of exam

– Skill/knowledge outcomes

– Student work (ie. Essays)

The stages of this evaluation will take place like this:

  1. Review/examine of academic grades
  2. Obtain data from a student centered perception of the course
  3. Obtain data from peer perception of the course
  4. Self reflection on the feedback and data

My first stage of the evaluation process is to obtain the qualitative data from the assessments and by using a varied form of SWOT analysis (Better Evaluation, n.d.), I will discover where the strengths and weaknesses are within the group, opportunities to develop particular areas and threats within the course. This information will then be compared with participant observation – are the participants who did not achieve good results actually showing up to class? Are they studying or reviewing the content?

The next stage of the student evaluation of teaching (SET) will be a cross between qualitative and quantitative data collection and will take the form of an evaluation document which will include survey and short answer responses. The use of SET is considered to be an important part of evaluation and also acts as a guide for any potential changes to course material and delivery (Shevlin, Banyard, Davies & Griffiths, 2000).

The criteria of this particular evaluation will address the student perception of the course, questions/statements such as:

– “Course material, media links and worksheets accessible”
– “Theory examination is manageable within the time constraints”
– “Students have access to the required readings”

Within each of the criteria question/statement, students will be required to respond to indicators and a rating scale, they will have an opportunity to provide a short answer comment to each criteria.

A sample is below:

Criteria (the question/statement)
Course material, media links and worksheets accessible
Rating Scale (out of 4)
1. Inaccessible;
2. Difficult to find;
3. Able to access but with difficulty;
4. Able to find material easily
Indicators (Yes or No responses)
– Media links work
– Course material easy to find on sharepoint
– Worksheets are targeted to the topic
– Course material relevant to weekly topics
Comment (open ended responses)

 

Peer evaluation is an important part of the overall process as peers are able to comment on the course material and resources, curriculum development and the aims, objectives and content. They will provide feedback on assignments, learning outcomes, assessment methods and addressing problems in a topic. (flinders website).

Recapitulation & Coda

So now I have the data and using the feedback from peers and the Student Evaluation of Teaching, I can reflect on how my peers and students perceive and experience the course and how effectively they are learning (Schwartz, n.d). The value of evaluation is improving performance and it helps identify the need to improve (Stronge, 2005). A trait of expert teachers is their ability to learn from reflection and observation (Stronge, 2005), in this case the student and peer feedback provides the catalyst for the need to continually improve and develop the course and materials.

 

 

References

Better Evaluation. (n.d.). Collect and/or Retrieve Data. Retrieved from http://www.betterevaluation.org/en/plan/describe/collect_retrieve_data

Danielson, C. (2013). The framework for teaching: Evaluation instrument. Retrieved from https://docs.education.gov.au/node/41101

Flinders University. (n.d.). What can peers evaluate? Retrieved from http://www.flinders.edu.au/teaching/quality/evaluation/peer-review/evaluate.cfm

Harvey, J. (1998). Evaluation Cookbook. Retrieved from http://www.icbl.hw.ac.uk/ltdi/cookbook/cookbook.pdf

Shevlin, M., Banyard. P., Davies, M., Griffiths, M. (2000). The Validity of Student Evaluation of Teaching in Higher Education: Love me, love my lectures?, Assessment & Evaluation in Higher Education, 25:4, 397-405, DOI:10.1080/713611436

Schwartz, M. (n.d). Self Evaluation of Teaching. The Learning and Teaching Office. Retrieved from http://www.ryerson.ca/content/dam/lt/resources/handouts/SelfEvaluation.pdf

Stronge, James H. (2005). Evaluating teaching: A guide to current thinking and best practice. Corwin Press.

Tran, N.D. (2015). Reconceptualisation of approaches to teaching evaluation in higher education. Retrieved from http://www.iier.org.au/iier25/tran.pdf

 

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s