Assessment matters: Designing your own tests

Introduction

In other articles, I’ve looked at quite a few issues surrounding assessment, including distinguishing between assessment and tests, assessing skills, diagnostic tests, assessing young learners and preparing students for tests and exams. However, recently, I’ve met more and more teachers who are required to design their own tests for their classes and yet have had no help in how to go about it. Therefore, the aim of this article is to examine some of the issues behind tests and to give some practical ideas and guidance to help teachers design tests that: 

• work;
• are fairly easy to administer;
• test what is wanted;
• are as fair as possible. 

One question that I’m often asked is: ‘How difficult can it be to design a test? After all, all you need to do is write some multiple-choice questions or a gap-fill!’ Well, actually, designing a test that is fair, tests what you want it to and is easy to administer is far harder than you might think. Often, you need to make a compromise, as it can be quite tricky to design a test that is easy to mark but is also reliable (more on this in the next section). Like anything else in teaching, designing good tests is a matter of practice and trying things out. 

What makes a good test?

The first thing is to think about the purpose of the test. Is it to distinguish between the students in the class? In other words, do you want some to do well, while others fail, so that it actually reflects their abilities and knowledge? Or is it to inform you (the teacher) of what needs to be re-taught because many, or most, of the students clearly don’t understand? Or is it purely for administrative purposes? If so, do you want the test to reflect well on the students and the teaching, or actually give a fairly accurate picture? 

These are just a few of the reasons that a test might be given. In many ways, what I am talking about here is the reliability of the test; that is, whether or not the results are consistent. For example, if two students who are clearly of different levels get similar scores, there is a question over the reliability of the test. 

Secondly, the test needs to reflect what has been taught – it needs validity. This might sound obvious, but I was once asked by a Ministry of Education to make a test harder by putting things into it that weren’t in the textbooks being used! Also, I don’t only mean language in terms of grammar and vocabulary, but in terms of activity types (more on this later). 

Thirdly, a test needs practicality – it needs to be easy to administer. For example, if a test takes hours to mark, then it’s not very practical for the teacher. 

One of the issues with designing a test is that sometimes there is a compromise between these three elements, particularly when practicality is taken into account. This can easily be seen when we look at the number of tests that rely on test items like multiple choice and gap-fill. After all, we may use these types of tasks in our teaching, but not to the extent that we can find them in tests. 

So, a good test needs to be reliable, valid and practical, and getting a balance between these three elements can be quite a challenge. 

There are plenty of ready-made tests out there. Why should I spend time designing my own?

One of the issues here is that any pre-prepared test is generic in nature. It’s written either to be used with a particular coursebook, or there is an expectation that the main focus of the course is taking the test at the end. This last point might appear a little strange, but there are plenty of courses where the focus is on passing a test (often one with international credentials) and the course focuses on the skills required to be successful in the test. (This is part of what is known as the washback effect, which I’ve talked about in another assessment article.) 

Of course, if you are using a particular coursebook and it contains ready-made tests, why not use them? But, before you do, consider the following points: 

• Did you cover everything in the book?
• Does the test cover everything you did?
• Did you use any other materials from other sources (for example, onestopenglish)?
• How long is the test? I.e. is the time it is meant to take longer than your lesson? 

In recent years, quite a few of the tests I’ve written for coursebooks have been in the form of editable Word documents. The reason for this is a realization by publishers that teachers do want to be able to match the tests to what they actually do in the classroom. However, one downside of this is that the reliability and validity factors can be compromised if the choices made by the teachers when editing result in a lack of balance. 

So one solution is to learn how to design your own tests and test items. Even if you continue to use ready-made tests, the skills you learn will help you if you do decide to edit the tests you have available. 

Some practical ideas

What are you testing?

The first thing to ask yourself is: ’What do I want to test?’ In most cases, you’ll want to test your students on what you’ve taught. This sounds obvious, but you have to think not only about the grammar or vocabulary you are testing but also the task type. If you introduce a task that your students have never seen before (or have used possibly once or twice but quite a long time ago), you are not testing their knowledge or use of the language – you are testing their ability to cope with a new task type! Is this really fair? So the first practical step you can take is to make a list of the types of tasks your students are familiar with. For example, are they used to doing gap-fill text and sentences? If so, do they usually have a box with the missing words given to them? 

Does it really work?

Let’s take the example of gap-fill texts. Have a look at these three short gap-fill tasks and decide: 

a. what is being tested;

b. if it works.  

Testing skills

So far, I’ve only really given examples with reference to vocabulary or grammar, but, over the past thirty years or so, teaching has become increasingly skills based. Almost everybody talks about the four skills; we’ve even divided some of this series up into these categories: reading, writing, speaking and listening. 

However, it can be rather misleading to treat the skills separately, as, in reality, they are usually taught, learnt, presented and used in combination. For example, if students have a reading text followed by a set of comprehension questions that require written answers, their writing ability is also being tested, not just their reading ability. Therefore, it’s important that we are aware of what skills are actually required when students are doing a task and that, if we are using the task for assessment purposes, we know we are assessing more than one skill. Of course, it is possible to assess one skill, but it is not easy. Look at these test activities and decide what is being tested. 

Conclusion

In such a short article, I’ve only really been able to touch on some of the main issues surrounding the designing of your own tests. You’ll find quite a lot of useful information in the other articles in this series – not only the ones on assessment, but the ones on topics such as reading, speaking, etc. 

In the main part of the article, I haven’t mentioned issues like criteria (although it is briefly mentioned in one of the comments), or what the best tasks for testing particular things actually are. What I have tried to do is get you to think about some of the key issues behind tests, so that if you are asked to write a test for your students you at least have some idea of what the pitfalls are and how you can try to avoid them. The main thing is, as with planning your lessons, to think about the aims, and the more practice you get, the better you will become.