Guide to writing up Research Skills 4 Practical 2

Guide to writing up Research Skills 4 Practical 2

The aims of the study

There are two aims of this study. One aim is to develop a questionnaire to measure attitudes towards free speech; the second is to determine whether respondents’ attitudes towards free speech differ according to their personality characteristics.

Aim 1: Questionnaire development

There are many ways in which research is conducted and the way that are you are probably most familiar with is the testing of a hypothesis derived from one psychological theory or other. In this case, there is a clear research question or prediction, e.g. we predict that participants in condition 1 will perform significantly more accurately that participants in condition 2. Though this is probably the most common way in which psychological research is framed, it is not the only one. With respect to the current aim, there is no research question or prediction as such. The focus is on the development of an instrument (in this case, a questionnaire) that measures a psychological variable (in this case, attitudes towards free speech).

The reasoning behind this practical was to give some idea of the process involved in the development of scales, questionnaires, and other psychometric instruments and the statistical techniques (e.g., item analysis, factor analysis) that are part of this developmental process. The actual process of scale construction is lengthy and is likely to involve several cycles of piloting and scale refinement before a final instrument is ready. Obviously we cannot replicate this in a short practical, but at least you will have some idea of what is involved so that you will be equipped to deal with more substantial problems in scale construction that you might encounter in the future (e.g. Year 3 project, PG research, or employment in any domain in which the measurement of public attitudes and opinions is a question of interest).

Although this aspect of the report (one that focusses on instrument development) may seem new to you, there are several papers exemplifying this type of research that are now on Minerva and which you can use for guidance when structuring your own report (Hense & Wright, 1992; Mehrabian, 1996; Zakrisson, 2005). As you will see from looking at these papers, a typical way of structuring the Method section would have a section on Participants; a section on the questionnaire itself and its composition (e.g. what was the format? How many items? How were they chosen? Were they developed from items used in previous questionnaires? This section may be called ‘Materials’ or ‘Scale Development’); and a Procedure section which deals with the actual distribution of the questionnaire and collection of data.

The above format, however, is only the most typical way of structuring the Method section, and you can adapt the Method section if you think that a modified structure would allow you to present the information in a clearer or more logical manner.

The justification for this aspect of the study is that most of the questionnaires created to measure attitudes to free speech, dogmatism, authoritarianism etc are now somewhat dated. Most of them were also developed in the USA and Canada. This being the case, they may not be ideally suited to the UK in the present day and so development of a new scale is worthwhile, especially considering the interest in the topic (e.g., https://www.gov.uk/government/news/landmark-proposals-to-strengthen-free-speech-at-universities; https://www.theguardian.com/education/2021/feb/16/proposed-free-speech-law-will-make-english-universities-liable-for-breaches; ).

Aim 2: Personality and attitudes towards free speech and censorship

The second aim of the study is probably more familiar to you, as it takes the form of making a hypothesis based on previous research. In this case, the hypothesis involves predicting how attitudes toward free speech will differ according to personality. Specifically, you will predict that respondents high in one of the Big 5 personality characteristics will differ in their free speech attitudes from respondents who are low in that personality characteristic. Which of the Big 5 you choose to focus on is up to you, but you should obviously refer to previous research in making your hypothesis or stating your research question. It may be the case that previous research on the personality dimension that you choose to write about does not allow you to make a strong one-way hypothesis. There may be mixed evidence, some of which points one way and some of which points in the other way. In this case, it is permissible to have a more open research question rather than a definite unidirectional hypothesis. Nevertheless, your framing of this more open type of research question should still be based on previous research and a recognition of its mixed findings. What is not permissible is an open research question that is not based on previous research (e.g. simply stating that there isn’t much research and so you are just going to explore possible links between your chosen personality dimension and attitudes to free speech). Such a purely exploratory approach does not allow you to demonstrate your ability to formulate a hypothesis/research question on the basis of previous research and so is something that should be avoided in the current report.

The way in which you structure your introduction and exactly how much attention you devote to each of these aims is up to you. We would, however, expect both aims to be given a reasonable amount of attention (i.e. don’t devote 90% of your Introduction to just one of these aims and refer to the other just as an afterthought – both need to be dealt with adequately).

Method

As mentioned above, in connection with Aim 1, the Method section of this sort of report might differ somewhat from that of the ‘classic’ format of an experimental psychology report in that it will include a section on Scale Development. You should also have sections on Participants, Design, and Procedure. The Design sub-section will deal exclusively with Aim 2 (Personality and attitudes to free speech) and the allocation of respondents to different groups depending on whether they score high or low on the personality dimension under investigation. As to Aim 1 (Questionnaire development), the normal Design terms (repeated measures, independent groups etc) do not apply to this aspect of the study because it does not involve the comparison of different groups or experimental conditions.

The investigation of Aim 1 used a survey design and the investigation of Aim 2 involved and independent groups design in which respondents were allocated to one of two groups (high and low) depending on their score on the personality dimension under investigation.

Other relevant information for this section:

The short personality questionnaire used is the Big Five Inventory-10 (BFI-10) (Rammstedt & John, 2007).

Ethical information: As this practical took the form of a classroom exercise designed to familiarise you with some of the aspects of questionnaire development, it did not require Research Ethics approval (as it does not constitute true research) and so there is no Ethics Committee number to quote. Nevertheless, the practical was consistent with British Psychological Society ethical guidelines. You should therefore state that the work followed BPS guidelines and that respondents were informed that they did not have to complete any part of a questionnaire that that they did not want to and that this would have no bearing on their successful completion of the practical module. This information was presented at the start of each questionnaire.

Results

The analyses that we would expect you to do are as follows:

1. Test of internal questionnaire reliability using Cronbach’s alpha and the subsequent identification of items that might be considered for removal in a future, more refined version of the questionnaire.

2. Factor analysis to determine the factors that may underlie attitudes towards free speech.

3. Independent groups t-test to see whether those who are high in your chosen personality characteristic differ from those who are low in this characteristic in terms of their attitudes towards free speech. Remember to present appropriate descriptive statistics.

For all of the above analyses, you will have to recode some of your variables – the reverse scored items for the free speech questionnaire and the BFI-10 will require recoding.

Presenting your results either in the text or in tables or in figures are all allowable, but one way may allow you to present your results in a clearer or more concise way than the others. It is up to you to decide the best method of presentation. You may find that one method of presentation is best for one of the above analyses, but that other methods of presentation are better for the other analyses.

In addition to the above analyses, you can do any other analysis or look at any other aspect of the data that you think would be useful in making sense of your results. Credit will be given if a student does present an interesting or useful additional examination of the data, but no extra credit will be given for additional analyses or examinations that do not really add anything to those mentioned above. You are therefore advised not to carry out extra analyses just for the sake of it – there has to be a point to your doing so.

Discussion

What you write about in this section will, obviously, be determined by your results but we would expect you to discuss the factors that seem to underlie the questionnaire responses, the possible exclusion of items in a future version of the questionnaire, and the relationship between personality and attitudes towards free speech and censorship. Here are some suggestions as to issues you might write about:

In the Results section you have, using Cronbach’s alpha, identified items that could be removed or reworded in a future, improved version of the questionnaire. In the Discussion you could take this further by considering what might be problematic about these items, e.g. are they worded ambiguously? Are they double barrelled? Do they contain double negatives or other features that might make them difficult to understand? Or does the item simply not tap into free speech attitudes in the way you thought. Would these items be excluded completely or could they be reworded to produce a better item.

In the Results section you will have identified and named the factors underlying the responses as revealed by your factor analysis. In the Discussion you could try to relate these factors to some of the other constructs (e.g. dogmatism, authoritarianism etc.) that have been discussed in the literature. Your factor analysis may end up yielding a large number of factors, some of which may be hard to interpret or account for very little of the variance. If this is the case you should feel free to concentrate on those factors that explain most of the variance and which make most theoretical sense. Do not feel that you have to discuss every single factor.

In the Results section you will have tested whether attitudes towards free speech differ between respondents who are high in your chosen personality characteristic and those who are low in your chosen personality characteristic. You could now try to relate these results to previous research looking at personality and relevant attitudes. Even if you cannot find literature specifically relating personality and attitudes to free speech, there is plenty of research linking personality with related topics, such as political outlook, dogmatism etc. that could potentially be included.

References

These should be in APA format

Appendices

You appendices should contain the following:

Reliability analysis

  1. Reliability Statistics – this is a small table containing Cronbach’s alpha and the number of items
  2. Item statistics – this is a table containing the means and SDs for each of your questionnaire items
  3. Item-total statistics – this is a table containing what Cronbach’s alpha would be if items were to be deleted

Factor analysis

  1. Table with KMO and Bartlett’s test of sphericity
  2. Table with Total variance explained by each factor
  3. Scree plot
  4. Rotated component matrix
  5. Table containing the items loading on each factor along with your chosen title for each of your factors (you will have to create this table yourself – it will not be part of your SPSS output)

t-test

  1. Group statistics table
  2. Independent samples test

All of these tables, apart from Factor Analysis table 5, will be in your SPSS output if you follow the same steps with the practical data as you did with the ‘practice’ data in the CBLs. Please do not include all your SPSS output – just include those items mentioned above.