Coaching and the flemish medical entrance exam: Efficacy and self-selection

• Author: Lars Vandekerckhove
• Position: Gent University, Belgium.
• Pages: 121-140

Page 121

1. Introduction

Twice a year, in July and August, the Flemish government organizes an admission test to the study of medicine and dental medicine. With this admission test, the Flemish parliament tries to avoid that students had to study for 5 or 7 years after which they could possibly be denied to practice their preferred profession, as there is a limitation on the enrollment of new physicians and dentists in Flanders. [9]

Research showed that participants prepare in a variety of ways for academic admission tests [4], [5]. One preparation activity that got special attention in previous studies was coaching, as it mostly is a paying activity in which a private tutor or tutoring agency promises a gain in results by attending their program.

However, these promises have often a weak empirical base. When investigating coaching effect, it is important to separate this effect from the individual differences that self-select participants into coaching programs. Otherwise, the perceived coaching effect is confounded with the effects of these individual differences on the test result. Eight possible self-selection variables were carefully selected from previous studies, which led to four hypotheses, questioning the link between these variables and coaching attendance. These eight variables are gender, previous education, nationality, relative study cost, self-efficacy, motivation and anxiety.

For variables such as gender, previous education and nationality, group membership is quite clear to the participant him- or herself. All of these variables arePage 122frequently quoted in research concerning the coaching effects on admission tests [15], [14], [10]. The amount of previous participations could affect the test result due to test and item familiarity, while it might also serve as a self-selection variable into coaching because of the dissatisfaction with the earlier results [4], [14], [8]. As individuals who make a smaller estimation of their odds to succeed, are probably willing to put more effort to increase these chances, the first hypothesis is:

Hypothesis 1: Individuals who, based on their gender, previous education, nationality or amount of participations, expect a lower chance to succeed, will more quickly turn to coaching.

One can also expect that more wealthy participants can more easily afford the high cost of coaching programs [7], [1], [14]. Hence the following hypothesis:

Hypothesis 2: Individuals who have fewer problems to bear the study costs, will more quickly turn to coaching.

Self-efficacy and motivation can be considered in the same assumption. Participants who approach the test with more trust and motivation, might be feeling this way because of their confidence with their preparation [15], [5]. Moreover, it is likely that highly motivated participants don't spare any effort to prepare in the best possible way [4], [10]. This led to another hypothesis:

Hypothesis 3: Individuals with a high score on self-efficacy and motivation will more quickly turn to coaching.

As a last self-selection variable, the participants' anxiety was included. Where Ryan et al. [15] dealt with it as a dependent variable, one might also assume that anxiety is an expression of stress-tolerance. Participants might choose for coaching in the hope of having enough confidence during the test administration, which led to the following:

Hypothesis 4: Individuals with a high level of anxiety will more quickly turn to coaching.

Reviewing the literature of coaching effects on similar admission tests to the FMAT, such as the Scholastic Aptitude Test (SAT) [13], [4], [14], [8], or the Medical College Admission Test (MCAT) [12] in North-America and the Israeli Psychometric Test (PET) [1] shows that coaching programs mostly have a rather small effect on the eventual test result. Thus, a small coaching effect on the FMAT can be expected, which led to the fifth hypothesis:

Hypothesis 5: Coached participants get a higher result than uncoached participants.

In an explorative part of this dissertation, the effect of online coaching on the FMAT is examined. The effectiveness of such programs has rarely been covered in research, and due to the ever evolving technology, studies from as far as ten years ago cannot be used as comparison [16],[12]. More interesting it is to compare the effectiveness of online coaching with traditional coaching programs. In this paper, online coaching is considered to be the consulting of commercial and free websites and bulletin boards by the participants, in preparation of the FMAT and its subtests. This description is broad but can be justified by the explorative nature and the recent development of the subject. The sixth hypothesis in this paper is:

Hypothesis 6: Online coached participants get higher results than not online coached participants.

Several studies noted that participants who attended a coaching program, already spent more time on preparation activities and this in an intensive way [5], [11], [13]. Therefore, it is interesting to examine whether the combination of both coaching and online coaching leads to a complementary, substitution or synergistic effect. If both concepts are complementary, it means both forms of coaching are independent of each other. If the coaching forms are substitutes, it would mean that the contents of the coaching programs and thePage 123online coaching programs overlap. A synergistic effect would imply that coaching attendance and online coaching strengthen each other when applied simultaneously. This translated into the following hypothesis:

Hypothesis 7: The effect of a combination of coaching and online coaching is larger than the sum of the separate effects.

2. Method

2.1. Sample

In 2008, 3214 individuals participated at the Flemish Medicine Admission Test. After this test, in February 2009, all participants received a questionnaire by e- mail. 40 participants gave an unreachable address. There were 965 respondents after the first mailing. The second mailing delivered 569 respondents. The third and last mailing added an extra 558 respondents. There were 1792 respondents in total (56.5% response rate). 1 respondent refused to participate and 3 respondents did not finish secondary school at the time of the test administration. 40 respondents only partly completed the questionnaire. Out of all participants at the FMAT, 55.2% provided useful data.

33.8% of the participants are male, 66.2% are female. In total, 21 nationalities can be distinguished. Most of the participants are Belgian (83.1%). The Dutch are strongly represented in the group with other nationalities, with 15.4%.

2.2. The Admission Test

The Flemish Medicine Admission Test is organized twice a year. In 2008, the first administration of the FMAT was on July 1st while the second administration took place in August 26^th.

The FMAT consists of two equally important parts: "Knowledge of and insight into sciences" (KIW) and "Acquiring and processing information" (IVV). The first part, KIW, focuses on biology, physics, chemistry and mathematics with an expected grade of difficulty around the average of the educational requirements for third grade secondary school (age 17 to 18). Each scientific subject arises in an equal amount of questions. The questions in the KIW part are mostly exercises. No mathematical proof or theorems are asked. Questions can be based on graphical representations, which also make the ability to analyze these representations important. Due to the nature of the questions, no difficult calculations are required. If necessary, data, formulas and constants are given. Participants can consult example items on the website of the FMAT. 10 out 20 points on the KIW test is the minimum required result to pass the FMAT.

The second part, IVV, tests the participant's ability to acquire and process information and the ability to solve problems based on given information. Thus, this second leg investigates the learning ability of the candidate, as the learning ability predicts future study results. This part is divided into a case study, with a reading test and a doctor- patient conversation, and a reasoning test. The reading test has a scientific theme. Participants have to read a text silently and have to answer related questions in which their assimilative capacity is tested. The doctor-patient conversation has a medical theme. Participants have to analyze a meeting between a doctor or dentist and a patient. In the conversation, a health issue of the patient arises. The participants are asked to make a fitting analysis and processing of the doctor-patient communication during these encounters. The reasoning test is designed to measure the participant's ability to process information. All acquired information needs to be processed mentally, to become problem solving. The reasoning testPage 124contains rigid time limits, as effective tackling of a problem is necessary in acute situations. Example items for all three subtests of the IVV test can be found on the website of the admission test. Example items for the reading test and the doctor-patient conversation can also be found in the information brochure. 10 out of 20 points on the IVV test is the minimum required result to pass the FMAT.

A result of 22 out of 40 points on the complete FMAT is the minimum requirement to pass. All questions are multiple choices; firstly, answers can be noted in a workbook, and afterwards, they can be copied on an answer sheet. To discourage guessing, correction for...