Mainly for MAs or undergrads who are short of prior research methods and stats training and just need to do whatever graphs and analysis of figures are necessary for their data. Obviously, since I don’t know exactly what each person will need to do for their data I can’t circulate a small but complete set of instructions that will suit everyone. I would have to give an entire SPSS course to cover every likely need (and I already do that in LG465/575 and 675). So I deal with people’s individual needs by email, my consulting hours, and in one or two collective MA dissertation training sessions in late term 2 and term 3.
If your research yields no figures, or too few of anything to be worth counting, and the account of results is to be primarily ‘qualitative’, then what follows this may still help you to organise your thinking a bit.
· Name and list for yourself all the things you have measured or recorded about the subjects
- all the different grouping categorisations
you put the subjects into (gender; year/level; on campus vs off; experimental
group vs control group etc.),
-
all the background things you got score-like figures for about each person
(e.g. proficiency score; number of languages
known…),
-
all the things you measured in each person for your research using different
tests or the like (e.g. ‘awareness of vocab’, with a set of test items; mean
response time to a set of items; score for network strength etc.),
- all the different
things you counted frequencies of in what people said or what you observed
(e.g. each different type of strategy you spotted in think aloud material; each
distinct type of reason for learning L2 that was mentioned by anyone in
interview; each type of error correction method you observed teachers using;
each sociolinguistic variable…),
- all the
different questions you asked in a questionnaire or the like which had yes/no,
or rating scale etc. answers (e.g. each attitude you got them to give information
on; each type of error correction you asked teachers about)
-
each comparison you wish to make between the people and themselves on the same
things e.g. at different times or in different versions/conditions (e.g.
reasons for learning now versus reasons they had earlier; awareness of grammar
vs awareness of vocab; teachers’ actual error correcting in class versus
what they say they do; performance in informal versus formal situation
etc…)
-
etc.
These will typically become the variables in your study, from the technical point of view. And typically each will form a column of figures in a data sheet, from which you can then draw graphs, and calculate any statistics you want (except the last type – the within-subject comparisons - which are entered as several columns). Each row normally is for one subject or case.
·
Now go through all those things - variables - and sort
them mentally into types. Primarily the ones that are explanatory (EV) or independent and the ones that are dependent (DV) or response variables.
Usually it is fairly obvious which variables you think might be having an
effect on or explaining others, and which might be on the receiving end, though
sometimes a variable can play both roles, depending on how you think of it.
Here are some examples of variables and the roles they often play… but beware,
variables are only explanatory or dependent in the context of particular
research… they are not necessarily one or the other always. You need to go back
to your research questions and/or hypotheses (which I hope you already
have clearly stated…!!!) to see clearly what role each variable is playing….and
indeed if you have measured some extra things you have no research questions
about, in fact. Well-worded RQs and RHs would typically mention all your key
variables explicitly. E.g. RH: ‘Male and female teachers differ in their
attitude to computers in English teaching’ = EV Gender; DV Attitude…
|
Often
Explanatory (Independent) EV |
Often
Dependent (Response) DV |
Involving
different subjects or subject groups (between subject comparisons)
L1 /
nationality of learner Gender of
subject Social
class of informant Age of
subject Level of
proficiency of learner Number of
languages learner knows Type of language
instruction the learner received in the past Teacher
versus student status of interviewee Length of
time subject has studied English Whether the
subject was in the experimental group or the control group Involving
the same subjects more than once
(repeated measures; within subject comparisons) Teachers’
actual in class error correction performance versus their self-reported
performance Real vs
role-played situation in which all subjects perform Purpose of
reading (to write summary vs do a comprehension test) Grammatical
vs lexical vs punctuation kind of awareness tested in all subjects The
experimental treatment vs the control treatment (where all subjects receive
both) etc. |
Rating of
attitude to a computer program Score on a
test of knowledge of the parts of speech Frequency
of hesitation Frequency
of use, or reported use, of the strategy of ‘guessing from context’ Score for
correct judgment of subject omission in error judgment task Average
length of turn in conversation Knowledge of
the meanings of a set of new words in a reading passage, from test score Mention or
not in interview of ‘liking the teacher’ as a reason for continuing to learn
L2 Number of
times each teacher corrected tense errors (as percent of how many tense
errors there were) Yes/no
response to a question Mean
response time on a set of similar stimuli % of times
each subject gets 3rd person –s
correct, out of possible instances in a recorded speech sample etc. |
·
Maybe you will find a few variables you recorded that
are not really either EVs or DVs. Perhaps they were recorded for a third
purpose, to check on something (control variables, CV). For example, if giving
a questionnaire and wanting responses only from native speakers of Arabic it
might be practical to include a question ‘What is your mother tongue?’ and
distribute the questionnaire widely. When you get the responses back you
actually exclude any non-native speakers of Arabic. Here ‘native language’ is
not an EV, as you are not comparing speakers of different languages; it is a CV
used to filter out subjects not of interest. Similarly you might include gender
not in order to compare genders, but just to see if you have a reasonable mix
of the two genders in your sample.
· If you seem to have a lot of variables not mentioned in your research questions, and not useful for any other purpose, you might go back and think if the questions need revising… otherwise why did you gather data on all those extra variables?
GET CLEAR EXACTLY HOW EACH VARIABLE IS SCORED/RECORDED
· Now think clearly about each variable's scale type. Is it in categories? If so is it two or more, logically ordered ones (ordinal) or unordered (nominal)? Or is it some sort of numerical score on a continuous number scale (technically on an interval scale)? (Avoid gathering data on any other scale type, e.g. rank orderings.) Typical examples:
|
Categories (
= Ordinal or Nominal scale) |
Scores ( =
Interval scale) |
|
Gender Age, if treated
as a set of age bands NS versus
NNS status of subject Social
class Grammatical
vs lexical vs punctuation kind of awareness of subjects Teacher vs
student Purpose of
reading (to write a summary vs to do a multiple choice comprehension test) Yes vs no
response to a question Responses
to a question where respondents indicate which one out of four types of
school they attended Mention
versus not of ‘to get a good job’ as a reason for learning English, in
interview Individual
ID of each subject Sentences
with relativised subjects as against ones with relativised objects, which
subjects respond to in a test Repetition method vs keyword method, as two ways of
remembering vocab that subjects are taught Three successive occasions on which data is gathered from
the same subjects etc. |
Years of
learning English Age, if
recorded as exact age in years Reading
proficiency score Rating of
attitude to RP, on a 1-5 scale in a questionnaire Number of
times each teacher corrects pronunciation, in observed classroom performance Score out of ten for correct naming of parts of speech in a
test Self-rating
of frequency with which each subject uses background knowledge when reading Score out
of five for correct labelling of five prodrop sentences as grammatical or not Number of
times a subject reports, or is observed, using a dictionary in a writing task % correct
score for getting the third person –s
right in a sample of spontaneous speech Mean
utterance lengths of children etc. |
Often you find that the explanatory variables are mostly in
categories and the dependent ones are mostly interval scores, but there is no
rule on that.
· Some work may be needed to reduce the data you gather to scores or categorisations at all. Some common special situations are as follows (though in some areas you may need supervision to do this properly). For any stats later, in general each variable has to be expressed and entered on computer as a set of scores or categorisations of each subject (case) separately, not group scores or % initially.
- Tests or inventories consisting of sets of items collectively measuring one thing. E.g. a 30 multiple choice item test of grammatical proficiency, or five items used collectively to measure instrumental orientation. Here you need each person’s total score to represent the variable (though for some more sophisticated stats it may be necessary to enter on computer the response of everyone on every separate item as a separate column). If some missed out some items, then you have to decide whether that counts as ‘wrong’ or just as missing. If the latter, then calculate each person’s score as an average or percent over those items they actually answered.
- Tests often measure several things at once with items relevant to each all mixed up. E.g. the computer presents ten words of each of three different types in randomised order and subjects respond as fast as possible pressing a key to indicate the word exists or not. Each person has to be given a mean response time for each of the three types of word separately. Each type of word would be a value of a repeated measures EV, and so be one column in the data sheet.
- If you included some items in a test just as distractors or fillers, to disguise what you were really testing, you obviously just leave those out of the count.
- Inventories used e.g. to measure integrative motivation often have several items whose responses have to be added to produce a score for the variable. However, some items will usually have been worded in such a way that e.g. the answer ‘yes’ indicates high motivation while others have been worded with reverse polarity so the answer ‘no’ indicates high motivation. When producing the total, one has to be careful to take that into account.
-
It is
usually acceptable, and easier, to treat responses on rating scales as if they were interval scores. So for a scale
‘always’ – ‘often’ – ‘sometimes’ – ‘never’, instead of reporting how many
people responded ‘always’, how many ‘often’, how many… etc. as categories you
treat each person as getting a score on a 1-4 scale or perhaps better 0-3,
treating ‘always’ as a score of 3, ‘often’ as a score of 2, etc. If, however,
the rating labels are clearly not equidistant, treat as categories (e.g. a set
like: ‘strongly agree’ - ‘not sure’ - ‘disagree’)
-
In questionnaires where
subjects are allowed to choose more than one of a set of options on an item,
then treat each option as a variable. E.g. if you offer them a set of reasons for
learning English and allow them to tick as many as they like, then each reason
is a category variable which they either say yes or no to. One questionnaire
question is not always equal to one variable and one column of data.
- Data from open questions, interviews, think aloud protocols, diaries, observation and the like often needs a lot of analysis in order to yield any figures. In such instances a lot of the work is in classifying what people said so that you can then count up instances of what is effectively ‘the same thing in different words’ said on different occasions or by different people. Usually you end up with frequencies for variables like how often each person used a given strategy, exhibited some behaviour etc. OR categorisations of people into those who mentioned something versus those who did not... etc.
- Where one counts, in spontaneous speech or written work, how often people use this or that variant of a vowel or produce 3rd person –s where required, usually there were not exactly the same opportunities for each person to use the targeted form. Hence it would be misleading to use the raw frequencies: it depends on your research questions, but usually one does not want to know that one person used more of something regardless of how many opportunities they had to use it. One typically gives each person a score in the form of the percent of the potential occurrences where they produced the form of interest. However, this can be misleading if there were few potential occurrences for some people (e.g. only 2, since producing the target form once gets represented as 50% just the same as for someone who used it 15 times in 30 potential occurrences). It is normal to set a lower limit and perhaps require at least 5 potential occurrences, or otherwise regard that person as producing no data for that variable.
- Where one counts how often people use this or that structure or strategy etc. in circumstances where there is no way of counting how often they could have used it, one may need to make some adjustment for length of task or the like. E.g. can one compare one student using 3 relative clauses (in a 220 word essay) directly with another using 5 (but in a 391 word essay)? Often it may be necessary to make such scores comparable by expressing them per 100 words, or per a unit of time etc. Strategy frequencies are often re-expressed as % of all strategies a person is recorded as using.
· Now run SPSS10 and enter your data. Start...programs...applications….
Choose to type in data and think of the grid as having one row per case. In your studies a case is usually a person: teacher, learner, native speaker or whatever.
And one column per variable or repeated measure, as described above.
· All your data has to be reduced to figures in columns for each case.... even category data. You have to get used to thinking of numbers in SPSS as sometimes standing for scores, sometimes for categories.
· Here is some imaginary (and rather nonsense) data to illustrate how data goes in.
The names of the columns/variables go where initially you see names like var00001. To change them click the tab at the bottom to go to Variable view and type a new Name. It cannot be longer than eight letters/numbers with no spaces and there are restrictions on what symbols you can use. Just pick some shorthand that triggers the memory. Under Label it is useful to also type the full meaningful name that you would like to give to each variable (e.g. Years of learning English).
In the Data View put only numbers in the columns, so any categories must get labelled arbitrarily as numbers. Type them at the top and hit the enter key to fill each marked 'cell' in the table. Or select a cell with the mouse and type in the number.
|
case_id |
gr_sp |
teach |
years |
oraltest |
t1_pr |
t2_unpr |
|
|
|
1 |
1 |
1 |
8 |
30 |
1 |
2 |
|
|
|
2 |
1 |
2 |
10 |
22 |
3 |
4 |
|
|
|
3 |
1 |
1 |
7 |
25 |
2 |
2 |
|
|
|
4 |
2 |
1 |
6 |
28 |
3 |
3 |
|
|
|
5 |
2 |
1 |
7 |
31 |
4 |
4 |
|
|
|
6 |
2 |
2 |
12 |
25 |
3 |
2 |
|
|
|
7 |
2 |
2 |
12 |
22 |
2 |
1 |
|
|
|
8 |
2 |
2 |
. |
19 |
4 |
3 |
|
|
|
9 |
1 |
1 |
6 |
35 |
2 |
2 |
|
|
These columns represent, from left to right:
- Each subject just numbered: that is there already in SPSS so no need to type
- A classification of learners into 1 Greek and 2 Spanish - category EV
- Taught by a native speaker or not (1=NS, 2=NNS) – category EV
- Years of learning English – interval EV
- Score out of 40 on a test of oral English – interval DV
- Two columns representing two values of a repeated measures category EV ‘task type’: task 1 was a prepared oral task, task 2 unprepared. In each column are recorded fluency scores of subjects, rated by a teacher on a scale 1-5 - interval DV.
· Note how groups of subjects are entered: never put scores for one group in one column and those for another in another beside it.
· Some imagined research questions for the study that led to the data gathering that yielded this imaginary data are:
In fact some of those could have been expressed as hypotheses.
Remember… all the variables should have some connection with the research questions or hypotheses…. Otherwise why did you gather that data????
· Note how the explanatory variables are usually entered on the left, though it doesn't actually matter what order the columns or cases are in, so long as the information is correct. However, there is another explanatory variable, the type of task, which appears differently because it is not a feature of different people; rather, all the subject do both tasks, so it appears in repeated columns - it is a repeated measures EV. In this instance, each column is not really a different variable, but the same one in different conditions (t1_pr, t2_unpr)…. But SPSS calls all columns ‘variables’ regardless.
· Note that subject 8 missed out providing the information for one variable, so SPSS records that as a dot. When entering your data there is a radical difference between someone scoring 0 for some variable and simply not providing any information, for whatever reason. Missing values should not be entered as 0.
· You can do usual things like highlighting and deleting rows, columns etc. (use the forward delete key), replacing numbers you mistyped.
· Make sure all the relevant information is entered about each column/variable. I already mentioned Name and Label for each column. In addition, for every column that contains numbers that label categories go to Values in the Variable View and fill in what the numbers stand for in words. Two columns are like this in the example. So for gr_sp you would click the Values cell, then click the grey square that appears and fill in the box that comes up. Enter Value 1 Value label Greek and click Add; then Value 2 Value label Spanish and click Add; then OK.
Do that for all the columns with category variables in them as needed.
Back in the Data View you can opt to see the data displayed with the value labels in words that you have entered. Click View in the top menu list and choose to tick Value labels. However, the real usefulness of doing all that will only appear later when you do graphs and stats, as SPSS will use this information to put more sensible labels on what it produces than it would otherwise.
· Save your own data for future use, as a .sav file. Print as you like. When you generate graphs and stats later, they will be stored and loaded as output files with suffix .spo, though any useful graphs and tables you make can also be cut and pasted into your writeup in Word.
NOW YOU ARE READY TO MAKE GRAPHS AND DO STATS….
..but before coming to me or my sessions you might think ahead what you want to find out from your data. Revisit your research questions and hypotheses. What do they imply for which columns you want to look at together or in relation to each other? What graphs would you like to make?
Make a list of things you are interested in the difference between or the relationship between.
Note, the above is a cheap and cheerful intro, overlooking many considerations and options one would find out about in a full course on research methods and stats and SPSS. It is in fact possible to do graphs and some basic stats also in Excel, and other packages. However, I go with SPSS since once the data is in it then it covers more or less any stats you might need. There is often more than one way of doing things in SPSS too, but I have chosen what I think is the simplest all-purpose way. SPSS10 is in all the public labs (and you can copy it onto your own computer free if you borrow the CD from the Computing Helpdesk on Square 2).
PJS May 01