Methods and principles from different scientific domains are, of course, usable in the learning sciences / educational psychology. One example is the scientific method. Often ascribed to Sir Francis Bacon (the 16th century natural philosopher and not the 20th century artist for those of you who are now going to Google® this or who need to disambiguate this for Wikipedia®) though it traces its history back to at least the 7th century BCE. We use it in many of the social sciences as a template or the ‘gold standard’ for how to carry out – or at least to a great extent plan and execute – social science research. The more I think about it, the more I get the feeling that the Heisenberg Uncertainty Principle – no need to Google this as I describe it below – also applies to education, learning, teaching and research on these topics.

The Heisenberg principle “ascribes the uncertainty in the measurable quantities to the jolt-like disturbance triggered by the act of observation itself”. In normal language this means that the act of measurement itself leads to uncertainty in measuring what is measured. Historically, the uncertainty principle has been confused with a somewhat similar effect in physics, called the observer effect, which notes that measurements of certain systems cannot be made without affecting the systems. In other words the act of looking a phenomenon influences the phenomenon being looked at. In science, this is often the result of instruments that, by necessity, alter the state of what they measure in some manner. A commonplace example is checking the pressure in an automobile tyre which is difficult if not impossible to do without letting out some of the air in the tyre and thus changing the pressure in it. This effect can be observed in many domains of physics.

Why am I bringing this up here? First off, research in social psychology has clearly shown that the presence of others can have an effect on and actually change one’s own behaviour (e.g., Aiello & Svec, 1993[1]). But what does this have to do with research in the learning sciences? As I stated, the randomised control laboratory experiment in cognitive and educational psychology – using the gold-standard scientific method – is based upon the idea that in the lab we can single out a to-be-researched independent variable, and as such, achieve results that are attributable only to that variable and are unaffected by other variables and factors. But is this so?

Spoiler alert!!!! No, I haven’t fallen off the bandwagon on or lost my faith in this approach to carrying out research and I’m still a staunch card-carrying advocate of randomised controlled trial research. I still think that this is the best way to begin; to make sure that the processes we hope to achieve with an intervention are actually due to the intervention and not some other extraneous influence. If this is the case, then the following steps bring us continually closer to ecologically valid, true educational settings where there is virtually no control. It’s only that I think that the absolutism that we often hear might benefit from a tiny grain of salt.

Take eye-movement research. This type of ‘objective online’ research, traditionally carried out in a highly controlled laboratory setting, allows us insights into how learners deal with (i.e., look at and process) texts, static illustrations, moving/dynamic images, hyperlinks, etcetera and any combination thereof. This technique enables us to ‘see’ what the learner is looking at down to the square millimetre / pixel, the time spent there standing still to take in the information (i.e., the eye fixations) and in moving to the next stopping point (i.e., the saccades and transitions) down to the millisecond. The power of this approach is made even stronger by the fact that we know, for example, that a person can be unaware that her/his eyes have moved (Belopolsky, Kramer, & Theeuwes, 2008[2]), and even when aware of the explicit need to look in a particular direction under certain circumstances, it can be difficult to do so (Munoz & Everling, 2004[3]; Theeuwes, Kramer, Hahn, & Irwin, 1998[4]). In other words, pure objective measurement. Right?

Maybe not. Are we actually measuring eye-movements that are only affected by the stimuli that we present in our ‘clean’ lab experiment or…? Risko and Kingstone, in their 2011 article Eyes wide shut: Implied social presence, eye tracking and attention[5], note that people often behaves differently when they know that they are being watched. In their study on whether social presence effects (i.e., effects caused by the fact that a viewer knows that (s)he is being watched by another) also influences that person’s ‘looking behaviour’. The researchers demonstrated that simply wearing an eye tracker, in their terms an implied social presence since the wearer knows/assumes that her/his eye-movements are being recorded and will be viewed by the experimenter both in real-time and at a later moment, can cause that person to avoid looking at particular stimuli (and possibly also to look at other stimuli). In other words, the implied social presence caused by wearing an eye tracker, was able to alter the wearer’s looking behaviour. This presents a methodological challenge to researchers using eye tracking. In their words:

The finding that wearing an eye tracker can alter looking behavior serves as an important reminder that the act of experimental observation can in some cases influence the behavior observed. At the most basic level, the present experiments demonstrate that (1) individuals are sensitive to their eyes being monitored (i.e., there is “eye tracker awareness”), and (2) participants are willing and able to modulate their looking behavior in response to this knowledge. (p. 295)

If this is the case in such objective, well-controlled situations, what then to think of less controlled learning and/or teaching situations?

Is there a solution to this problem; a way to negate this social presence effect? Apparently the ’fly-on-the-wall’ tactic used by film makers – from serious ethnological documentary makers to the makers of emo-TV reality shows – might be just the answer. If you film people long enough, the people being filmed will forget that there is a camera rolling. For the researcher, this means that (s)he can begin the experiment, but must wait a while before actually tracking and recording the eye movements. We know this because Nasopoulos, Risko, Foulsham and Kingstone (2014[6]), in a follow-up study, showed that while this social presence effect takes place when a person is fitted with an eye tracker, in less than 10 minutes this effect ebbs away to zero. However, when the attention of the wearer is drawn back to the eye tracker, the social presence effect is immediately reactivated. In other words, the social presence effect is a transient effect; good news for researchers who use eye trackers to measure attention and behaviour.

Are there other examples of such ‘observer effects’? Closely related to what I just discussed is the experimenter effect, an effect caused by actions of an experimenter. A typical example is when the experimenter knows who has received what treatment or intervention as this might cause the experimenter to see and interpret phenomena (e.g., behaviours, processes, etcetera) in the experimental or the control group where none really exist. I discussed this earlier in a blog which I called 20/20 Blindsight or There is none so blind as (s)he who will not see[7] when I discussed the poisonous mixture of confirmation bias and hindsight bias. Rosenthal called this an expectancy effect. In an ingenious experiment he found that psychology grad-students typically found that a group of rats was faster to learn its way through mazes when the students were told that one group was “brighter” than another.

Another example is when the experimenter gives respondents subtle cues or signals that may affect their performance or responses. The cues may be unconscious nonverbal cues, such as muscular tension or gestures or vocal, such as tone of voice. Research has demonstrated that the expectations and biases of an experimenter can be communicated to experimental subjects in subtle, unintentional ways, and that these cues can significantly affect the outcome of the experiment (Rosenthal, 1998).

The solution here is via double-blinded experiments which is ‘normal’ in research in the fields of pharmacy and medicine where the field normally requires it and where there is usually the funding available for the extra assistance that is needed, but not in psychology and/or the learning sciences (as well as in most other social sciences) where funding is limited and the field does not traditionally require it. The social sciences typically does not go further than a single-blind research design where the participants are not aware of whether they are or are not in an intervention condition or a control condition. Unfortunately, this not the case for the experimenter. In general what we see in single-blinded experiments is that the experimenter – hopefully – randomises the participants, but having done this knows who is in which group, often is the one who carries out the intervention treatment (oh the subtle and even not-so-subtle cues that might be given off here!), is the person recording, interpreting and evaluating the responses, gestures or utterances of the respondents (often with the aid of a coding scheme / protocol and the coding of a percentage of the data by a second rater for calculating the inter-rater reliability), and then is the one who analyses and interprets the results. While a double-blind procedure minimises biases on the part of the experimenter, typical social science research – using a single-blind procedure – compounds the biases, especially in qualitative research!

In other words what Werner Heisenberg said about nature is possibly universal[8]:

What we observe is not nature itself, but nature exposed to our method of questioning.

Natural science does not simply describe and explain nature; it is a part of the interplay between nature and ourselves; it describes nature as exposed to our method of questioning.

[1] Aiello, J. R., & Svec, C. M. (1993). Computer monitoring of work performance: Social facilitation and electronic presence. Journal of Applied Social Psychology, 23, 537-548.

[2] Belopolsky, A. V., Kramer, A. F., Theeuwes, J. (2008). The role of awareness in processing of oculomotor capture: Evidence from event-related potentials. Journal of Cognitive Neuroscience, 20(12), 1–13.

[3] Munoz, D. P., & Everling, S. (2004). Look away: The anti-saccade task and the voluntary control of eye movement. Nature Reviews Neuroscience, 5, 218–228.

[4] Theeuwes, J., Kramer, A. F., Hahn, S., & Irwin, D. E. (1998). Our eyes do not always go where we want them to go: Capture of eyes by new objects. Psychological Science, 9, 379–385.

[5] Risko, E. F., & Kingstone, A. (2011). Eyes wide shut: Implied social presence, eye tracking and attention. Attention, Perception and Psychophysics, 73, 291–296. doi:10.3758/s13414-010-0042-1

[6] Nasiopoulos, E., Risko,E. F. Foulsham, T., & Kingstone, AQ. (2014). Wearable computing: Will it make people prosocial? British Journal of Psychology, XX, XXX. doi: DOI:10.1111/bjop.12080

[7] Also to be found at

[8] Heisenberg, W. (1958). Physics and philosophy: The revolution in modern science. New York: Harper & Row Publishers.

Join the conversation! 8 Comments

  1. Dit is op From experience to meaning… herblogden reageerde:

    An interesting piece by Paul Kirschner (co-author of our new book on Urban myths about education and learning): “While a double-blind procedure minimises biases on the part of the experimenter, typical social science research – using a single-blind procedure – compounds the biases, especially in qualitative research!”

  2. Interesting! I created a much more simple, less informed version along the same lines here:

  3. Double blinds are used in medicine becasue patients anticipate getting better when they recieve a treatment, in most cases education studies will measure attainment, it seems unlikely children will deliberatly attain better becasue they know they are recieveing some programme

    • Though it may seem likely to you, it is the case. But an even larger problem is that the researcher anticipates, sees and interprets things differently if (s)he knows who received a treatment and who didn’t. This is also the case in medicine!

  4. Great piece, Paul. As someone who is continuously experimenting with teaching methods, I wonder how the effects of these interventions can be validated in my daily practice. Normally, teachers have neither access to funding nor the time to perform randomised controlled trials. So, the uncertainty effect is in full daily swing. How, then, can we draw at least half-way reliable conclusions from our classroom experiments?
    As you rightfully said, it is proper to add a few grains of salt to some of the bolder education scientific conclusions. I am still struggling to reconcile some principles that seem to be based on solid experimental evidence with the endlessly varied conditions in the actual classroom. Some appear to work fine, as science predicted, while others, which science (e.g., you) rejects, work also fine, at least in my classroom.
    Although, as a scientist, I welcome any attempt to understand, both qualitatively and quantitavely, I am also filled with wonder about the unpredictability and inherently unquantifiable nature of the process of teaching. Forgive me. I am hopelessly romantic.

  5. Dick,

    Thanks. All experience is context-bound and the context of education is very different from the context of the cosmos. This does not mean that educational science should not look for theories, generalities, etc but rather that when theories are not rejected (not the same as confirmed) it only meas that the probability of something working is greater; not absolute.


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

About Paul Kirschner

Nederlands: Prof. dr. Paul A. Kirschner, dr.h.c. is Universiteishoogleraar en hoogleraar Onderwijspsychologie aan de Open Universiteit. Hij is ook Visiting Professor Onderwijs met een leerstoel in Leren en Interactie in de Lerarenopleiding aan Oulu University (Finland) waar hij ook een Eredoctoraat heeft (doctor honoris causa). Hij is een internationaal erkende expert op zijn gebied en heeft zitting gehad in de Onderwijsraad in de periode 2000-2004 en is lid van de Wetenschappelijk Technische Raad van SURF. Hij is Fellow of the American Educational Research Association (AERA; NB de eerste Europeaan aan wie deze eer werd toegekend), de International Society of the Learning Sciences (ISLS) en van de Netherlands Institute for Advanced Study in the Humanities and Social Science of the Royal Dutch Academy of Sciences (NIAS-KNAW). Hij was President van de International Society for the Learning Sciences (ISLS) in de periode 2010-2011. Hij is Hoofdredacteur van de Journal of Computer Assisted Learning en Commissioning Editor van Computers in Human Behavior, en hij is auteur van Ten steps to complex learning (Routledge/Erlbaum). Hij schrift ook regelmatig voor Didactief (de kolom KirschnerKiest over wat docenten kunnen met wetenschappelijke resultaten). Hij is ook medeauteur van het boek Jongens zijn slimmer dan meisjes XL (EN: Urban Myths about Learning and Education). Hij wordt gezien als expert op veel gebieden en vooral computerondersteund samenwerkend leren (CSCL), het ontwerpen van innovatieve, elektronische leeromgevingen, mediagebruik in het onderwijs en het verwerven van complex cognitieve vaardigheden. English: Paul A. Kirschner (1951) is Distinguished University Professor and professor of Educational Psychology at the Open University of the Netherlands as well as Visiting Professor of Education with a special emphasis on Learning and Interaction in Teacher Education at the University of Oulu, Finland where he was also honoured with an Honorary Doctorate (doctor honoris causa). He was previously professor of Educational Psychology and Programme Director of the Fostering Effective, Efficient and Enjoyable Learning environments (FEEEL) programme at the Welten Institute, Research Centre for Learning, Teaching and Technology at the Open University of the Netherlands. He is an internationally recognised expert in the fields of educational psychology and instructional design. He is Research Fellow of the American Educational Research Association and the Netherlands Institute for Advanced Study in the Humanities and Social Science. He was President of the International Society for the Learning Sciences (ISLS) in 2010-2011, member of both the ISLS CSCL Board and the Executive Committee of the Society and he is an AERA Research Fellow (the first European to receive this honour). He is currently a member of the Scientific Technical Council of the Foundation for University Computing Facilities (SURF WTR) in the Netherlands and was a member of the Dutch Educational Council and, as such, was advisor to the Minister of Education (2000-2004). He is chief editor of the Journal of Computer Assisted Learning, commissioning editor of Computers in Human Behavior, and has published two very successful books: Ten Steps to Complex Learning (now in its third revised edition and translated/published in Korea and China) and Urban Legends about Learning and Education (also in Dutch, Swedish, and Chinese). He also co-edited two other books (Visualizing Argumentation and What we know about CSCL). His areas of expertise include interaction in learning, collaboration for learning (computer supported collaborative learning), and regulation of learning.


English, onderwijs, onderzoek


, , ,