Footnote 8 In many developed countries the selection of staff for key positions in industry, commerce, and government service involves the use of objective assessment techniques. These techniques include testing of intelligence, aptitude, and attainment; questionnaires to cover interests, motivation, and other aspects of personality; and behavioural exercises which simulate situations typical of the job content. These data are then co-ordinated and used in combination with the result of an interview to ensure, as far as possible, that the job and the individual are appropriately matched. Such techniques have not been applied in any substantial way to the selection of doctors for specialist training or practice, although they have been used in some centres to assist in medical student selection.

In a speciality such as surgery which is increasingly concentrating on the intellectual or academic, and scientific, aspects of its work it is surprising that more attention has not been paid to selection of recruits.

Aptitude for surgery may be tested by putting the individual into a training programme and then assessing the result, as has largely been the case up to the present. This approach is based on the apprenticeship system of the past and is probably no longer appropriate. Any training programme will contain some individuals found wanting in certain qualities: the use of selection techniques before entry aims to minimize their number. Preliminary testing will not avoid the problem altogether. However, the escalating costs of training surgeons in an increasing number of specialties dictates, even if only at an economical level, that when there are too many aspirants to a surgical career, as there are in many parts of the world, it is better that only those with appropriate ability should be selected to train. The cost of training is important, especially when considered in relation to diminishing overall resources for medical education, clinical treatment, and research. There are, however, other important factors: personal costs to the trainees who prove to be inadequate to the demands of their chosen career, to say nothing of the impact on their families, and the disruption of the training programme itself with the attendant difficulties for trainers whose task it will be to guide their junior colleagues into another branch of medicine.

In the United Kingdom the procedure for selection into many surgical training programmes may be described as somewhat haphazard. Initially, self-selection plays pits part: the aspirant knows that the preliminary surgical examination must be passed. Hopefully before embarking on this course of study the individual will have sought advice from his previous surgical teachers, but this is not a requirement. Academic ability has, rather worryingly, been shown not to correlate well with later surgical performance. Academic ability, however, usually plays an important role in selection for entry into a training programme, in conjunction with the candidate's curriculum vitae and the impression gained at later interview.

There are disadvantages inherent in the usual forms of the curriculum vitae and interview, both of which are often unstructured in their format. These disadvantages will be discussed below but to a large extent they may be overcome by the use of more formal assessment procedures.

All assessment techniques should meet three main criteria: they must be technically sound, i.e. based on a sound rationale and be demonstrably valid; they must be economically feasible in respect of cost benefit, including calculation of time involvement; and they must be politically acceptable, both to the instigators of the test procedures and the candidates themselves.

The factors which must be covered in any comprehensive procedure make up the ‘seven-point plan’, namely education, work experience, circumstances, physical make-up, abilities, personality, and motivation. The first three factors, and to some extent the last, lend themselves to assessment by structured questionnaire, and the fourth requires a medical examination. The principal difficulties in assessment lie in the categories ‘abilities’ and ‘personality’, and it is in these spheres that objective testing may prove particularly useful.

It has to be emphasized that the result of all such tests must be considered in conjunction with the interview. The test results are not designed to be more than complementary to this final interview procedure.

The validity of any test may be expressed as a correlation coefficient ranging from 0 to 1, the size indicating the degree of relationship between scores of assessment and the given criterion of success at the job: 0 indicates no relationship and 1 a perfect relationship. A coefficient of less than 0 indicates an inverse relationship between the test measurement and performance.

The validity coefficient of any test or assessment procedure should be 0.3 or above for the test to have credibility. The best scores are achieved using ‘assessment centres’ which combine several different assessment tools, but these may be both time-consuming and expensive to administer (score 0.4 - 0.6). Ability tests alone produce a range of 0.2 to 0.5, and interviews 0 to 0.3, the higher score relating to structured interviews with interviewers who are familiar with interview techniques and who ask questions with a known discriminatory value. The unstructured interview which is all too frequently used, when random questions are asked by untrained interviewers, has no more power to predict later performance than has random selection. Such interviews therefore have a low validity and low reliability: different interviewers reach different conclusions about the same candidate when their recorded assessments are later compared. The initial impact made by the candidate on any one interviewer may be overly influential, due to a wide variety of factors. This initial impression tends to set up a self-fulfilling prophesy as the interviewer asks questions of varying difficulty in a set manner. Interview questioning based on a designed personality questionnaire, with the applicant's answers, goes some way to overcoming this problem.

As a test measure in its own right, however, the personality questionnaire has in the past often scored poorly, a figure of 0.15 being quoted. Since questionnaires have usually formed part of any battery of formal test procedures the whole concept of such procedures has been called into question. Such criticism fails to take into account the fact that there are both poor and good personality questionnaires. The latter includes the more recently available occupational personality questionnaires, produced with a specific occupation, or occupations in mind. To design such questionnaires an assessment of the job, or task, under consideration has first to be performed—the so-called task analysis—and this will require the professional advice of occupational psychologists.

Although the validity of the personality questionnaire depends on the self-awareness and honesty of the respondent available data show it to be a surprisingly reliable method, the validity co-efficient rising to over 0.3. The questionnaire may have a further advantage which was demonstrated by surgeons in the Netherlands who adopted this system. From 1983 all those contemplating a surgical career in that country were required to complete a questionnaire, which by its questions demonstrated much about the nature of surgical training, and surgery as a career. In the first year only 52 per cent of questionnaires were returned, the percentages in the following 4 years being 70, 80, 56, and 49. The questions provided a good test of motivation, confirming the value of this method as a selection tool.

Another aspect of the individual's personality that may be assessed is how others see him or her. The opinions of friends or colleagues may be used to complete what may be called a personality rating scale, which gives a more complete picture of an individual. Some interesting differences of perception may emerge: these may have considerable relevance to job selection or training potential. For example, individuals may not realize how they communicate their thoughts and emotions, or they may not know that others perceive them as having skills such as those of leadership of which they are unaware. Although such information must be sensitively sought, and used, it is in fact only a logical development of the ‘testimonial’ or ‘reference’ which has long been used in surgical selection procedures. Being structured, that is seeking information by asking questions of known relevance to the occupation, this assessment method may reasonably be expected to overcome some of the recognized shortcomings of the written reference. These shortcomings range from the omission of important information, for example regarding technical skill or personal relationships, to damning by faint praise!

Reference to technical skill raises the question of whether it is possible objectively to test actual or potential operative ability in a surgical trainee. My interest in aptitude testing was initially centred in trying to find methods of testing what I thought of as manual dexterity. As so often proves to be the case, this issue proved to be considerably more complex than expected. Tests of manipulative skill have existed for many years but they are unsophisticated and test rather simple tasks which have, in fact, been shown to have little relevance to surgical skills. Such a test involves picking up pegs and putting them in a pegboard: the greater number inserted in the given time the greater the dexterity score. Another test introduces the perceptual aspect of dexterity when the requirement is to insert pieces of wood or plastic of different shapes and sizes into spaces in a template, rather like a complicated jigsaw puzzle. Stress factors may be simulated in several ways: by time limitation, adding other afferent stimulation such as noise, or by making relatively slight differences in the shape and size of some of the pieces.

What surgeons think of as technical skill or dexterity is more complicated than either of these tests can demonstrate. Visuospatial organization and stress tolerance are the most important factors in surgical skill. Visuospatial organization involves hand/eye/brain co-ordination: a good example of this is shown in the ability of a competent fibreoptic endoscopist to co-ordinate the action of the hand and fingers to allow analysis of the image from the distant area being visualized. Some individuals starting training have this facility almost as an innate ability, others appear to have difficulty even after prolonged training. This is perhaps not surprising, for it is widely accepted that some individuals have mathematical, or language-learning ability which others lack. An otherwise intelligent and capable person should feel no sense of inadequacy in lacking such specific abilities, the defect—if such it be—will almost certainly be compensated in other directions. At a time when endoscopic and minimally invasive surgical techniques are becoming ever more important aspects of surgical skill it does, however, seem wise to attempt to determine a trainee's potential in visuospatial skill by initial objective testing.

The scoring of such testing may be weighted according to the criterion analysis for the specialty. For example the test for visuo-spatial organization might carry a higher score for the prospective urologist who will increasingly be involved in endoscopic surgery than for the accident surgeon who will be treating patients with major trauma. If testing were performed at entry into surgical training counselling concerning specialty choice within surgery would be easier, and there would perhaps be less chance of later disappointment for the trainee if he or she were unsuccessful. Even more important, the chance of a less than fully competent individual passing through a training programme would be minimized. In these circumstances, whether the latent defect is of technical skill or personality, stress for the individual increases. This in turn may set up a vicious circle in which further deterioration in clinical technical performance occurs and at worst the individual becomes insecure, depressed, or resorts to the supposed benefits of drugs or alcohol.

Before describing the initiative which has been taken by the Royal College of Surgeons of England in exploring the use of personality and aptitude testing in the selection of surgical trainees it is appropriate to mention the methods used by the Royal Air Force in officer and air crew selection. Before the second world war air crew selection techniques were not dissimilar to those which are used in many surgical centres today. A group of senior officers met, considered the applicant's curriculum vitae, and then held an interview. Following mobilization there was an unhappy incidence of flying accidents—clearly they had not always chosen those with aptitude for flying. Methods of selection which were more accurate were then sought, and for the past 50 years these methods have been developed and refined, more recently with the aid of computerized tests and scoring. The Air Force has had an unrivalled opportunity of validating its initial selection procedures by following those selected through their training, at the end of each stage of which there is a formal assessment. The stages are initial officer training, basic flying training, advanced flying training, and then after ‘graduation’ into squadron service, which again carries a requirement for formal annual assessment. By modifying the selection techniques as a result of this continual validation exercise, and using criteria weighted as appropriate for the tasks involved a ‘P’ (for pilot) or ‘N’ (for navigator) score has been produced from the results of the initial selection procedures. These scores prove to be highly significant in predicting success in basic flying training. In a sample of 805 those having a ‘P’ score of 80 to 89 had a 90 per cent chance of success whereas only 60 per cent were successful if their initial score was 50 to 59. It has to be said that the initial selection procedures are conducted over a 3-day period and involve not only personality, aptitude testing, and an interview, but performance in problem solving in a team, both as leader of the team and as a member of it. This is an example of the ‘assessment centre’ testing referred to above. Such comprehensive testing is increasingly used by industrial concerns for selection of trainees for senior staff positions. It might be thought that this was a more appropriate and comprehensive method than that usually used to select future surgeons.

In Air Force terms, the cost of one lost aircraft, let alone the cost of training an individual who later fails as an aviator, makes the expenditure on the selection process worthwhile. In surgical terms the benefits to patients, and providers, of employing a surgeon whose clinical audit figures later show a low incidence of morbidity and mortality are likely to be equally impressive.

In 1987, after hearing the arguments in favour of a more objective approach to the selection of surgical trainees, the Council of The Royal College of Surgeons of England organized a symposium at which the principal speakers were an occupational psychologist, the Commandant of the Royal Air Force Officer and Air Crew Selection Board, and two surgeons representing the Netherland Association of Surgeons, with their occupational psychologist colleagues, who had adopted these techniques.

Following this it was decided that a formal feasibility study should be conducted and with the aid of a grant from the Department of Health this was started by a group of occupational psychologists in 1989.

There are two reasons for describing in some detail the nature of this task analysis. First, its findings will provide the basis for the design of questionnaires and tests of personality and aptitude. Second, it is important that those members of the profession who are sceptical about the value of more formal selection procedures should be made aware of the basis of such procedures so that they might reassess their opinions.

A combination of three formal analysis techniques, and two informal methods, was used to identify key attributes and criteria for success in surgery as viewed by those already in the specialty, and those working with surgeons.

This is an integrated job analysis system. Information about surgery as an occupation was collected in a structured way from questionnaires. This information was then analysed by computer to meet a variety of objectives, including a profile of job tasks and a profile of the attributes required to enable an individual to fulfil these tasks. Six surgeons (one consultant, two senior trainees, and three junior trainees) completed the existing professional/managerial version of a work profiling questionnaire, which comprises a check list of 344 activities grouped into 32 categories. The respondents were required first to select the eight categories most relevant to achieving the job objectives. Second they were required to indicate, for each item within each of the selected categories, the time spent in this activity and its importance in achieving the job objective.

This technique is used to provide a structure for eliciting the attributes which distinguish between good and poor job incumbents. It has the important feature of not offering respondents preset dimensions for response and allows them to draw on their own experiences to identify salient features of the job performance. It is also particularly useful for drawing out subtle, yet often very important, aspects of job performance which may not have been previously considered.

Interviews were conducted on a one to one basis with 10 surgeons (six consultants, four senior trainees) each of whom was in a position to make comparisons between 10 known surgeons, half of whom were considered to be good performers, and half considered to be less good. Comparisons between these opinions were made, thereby eliciting characteristics considered necessary for good performance.

The critical incident technique is a procedure for collecting observed incidents where the outcome has proved very important or critical in terms of achieving job objectives. The emphasis is not upon routine activities but rather upon those essentials in performance which make the difference between success and failure.

Critical incident interviews were conducted with a total of 44 surgeons in groups of between four and eight persons (one group of consultants, one group of senior trainees; six groups of junior trainees). They were asked to relate incidents from their own experience which were either successful or unsuccessful in fulfilling job objectives. These were then analysed to provide a composite picture of job behaviours, and the skills, qualities, and attributes necessary to fulfil the tasks and objectives effectively.

In addition to the structured techniques outlined above less formal interviews were conducted with a sample of theatre staff (anaesthetists, theatre sisters, etc.) and patients, as well as with the surgeons themselves. In addition, the activities and behaviour of surgeons was observed by the psychologist during the course of operations, clinics, and ward rounds.

A total of 60 surgeons (47 general surgeons and 13 orthopaedic surgeons) was interviewed. The 10 who completed repertory grids were each in a position to comment on the behaviour and attributes of 10 other surgeons well known to them. Thus, in all, the behaviour and attributes of some 150 surgeons were sampled. Some of these were still in general training and planning to enter other specialties at a later date. No significant differences were observed between the ‘general’ surgeon group and the ‘orthopaedic’ surgeon group.

The 60 surgeons interviewed included representatives of both district and teaching hospitals located in four different areas of the country. Women were grossly under-represented in the sample, reflecting the situation in surgery generally. All participants were volunteers, the trainees having been encouraged to take part in the exercise by their consultants.

The surgeon's role, regardless of specialty, is clearly multi-faceted. The major task categories identified were theatre work, ward rounds, liaison with other medical staff, outpatient clinics, research, teaching, and administration.

Key objectives which emerged repeatedly in the data were total patient care, the advancement of surgery both through teaching and research, and effective communication and efficient management of resources in order to achieve the other two objectives.

Twelve criteria were identified by the occupational psychologists as important for surgeons in general. In order of importance these were:

(i)interpersonal skills

(ii)communication skills

(iii)responsibility and leadership skills

(iv)evaluative and analytical skills

(v)broad and balanced perspective

(vi)decision making skills

(vii)personal organizational skills

(viii)stress tolerance

(ix)self-motivation

(x)political awareness

(xi)self-insight and integrity

(xii)basic skills and abilities:

(a)basic academic ability,

(b)technical competence (including manual dexterity, good eye/hand co-ordination, spatial skills, and capacity for focused and sustained attention).

In addition to the general themes outlined above, the data also highlighted some apparent conflicts in the role requirements of surgeons, both in terms of the demands made at any one time and in terms of the requirements of different stages of the surgeon's career. Conflicting demands within individuals included the need to balance the requirements for logical versus lateral thought, and an academic versus a clinical focus. Consultant versus trainee differences included quite dramatic shifts in the need to be co-operative as opposed to competitive, challenging as opposed to conformist, and accepting of total responsibility as opposed to responding to directions.

All of this implies the need for flexibility and preparedness to change on the part of recruits for and in surgical training. Extremists in any direction are unlikely to be able to cope with these conflicting demands.

Key environmental differences that could render some individuals more suitable for a teaching hospital as opposed to a district hospital career emerge from the study. The degree of formality was commented upon, the hierarchy being seen as less formal in district hospitals. Similarly, an academic as opposed to clinical emphasis was identified, district hospitals being seen as more patient-orientated. Finally, teaching hospitals were seen to be more divided in their activities as opposed to the integrated approach perceived in district hospitals.

Interview data also highlighted a number of imminent developments and changes which seemed likely to increase the demand upon surgeons in the foreseeable future. Key trends identified were changes in the National Health Service, surgically related advances, and changes in information technology. Additionally it was considered that patients' attitudes were changing, becoming more sophisticated about the surgeons' role consequent upon increased specialization. These changes suggest a likely increase in the need for certain skills and abilities over and above those already identified as important for surgeons, including entrepreneurial skills, the ability to manage computer technology, and the ability to analyse financial data. Similarly an increase in managerial and organization skills, greater flexibility and tolerance, and the capacity to adapt readily to change were regarded as essential.

The findings from this study which complement previous studies relating to the selection of surgical and anaesthetic trainees are being validated. In excess of 200 surgeons randomly selected from the fellowship of The Royal College of Surgeons and the specialist surgical associations, are voluntarily co-operating in the validation exercise. When completed this exercise will therefore provide a sound basis for questionnaire and test design. Later the validation procedure must continue, however, using the result of in-training assessments of those who have initially been tested, in the same way as the Royal Air Force has done. There is little doubt that greater use of in-training assessment, using informed criteria, should become routine rather than occasional. At all stages the results of such assessments should be discussed with the trainees themselves. At various stages of the in-training programme account will have to be taken of those aspects of required surgical ability—for example, managerial skill or the ability to analyse financial data relevant to clinical practice, which have been identified by the task analysis as being important. This has implications for the structure of the training programmes.

Initially the questionnaire and test batteries might be offered to senior medical students or junior doctors who are contemplating a career in surgery. The availability of such ‘tests in advance’ might well attract into the specialty some able young people who at present shun surgery as a career. In the United Kingdom the existence of an initial entrance-to-surgery examination, the Fellowship, will continue. Just as cognitive skills are tested by multiple choice questions and the marks used in overall assessment so it may prove appropriate to use the marks produced by tests of personality and aptitude.

Surgery as a specialty within medicine is changing. Increasing specialization is resulting in new technical and personal demands on the surgeon who, in addition, is exposed to an increasingly litiginous environment. For the sake of surgery itself, the individual surgeon, and most importantly the patient, the selection of those who will carry the specialty into the 21st century is of critical importance. This chapter points to some ways in which present selection methods might be improved.

Gough MH, et al. Personality assessment techniques and aptitude testing as aids to the selection of surgical trainees. Ann R Coll Surg Engl 1988; 70: 265 - 79.

Graham KS, Deary IJ. A role for aptitude testing in surgery. J R Coll Surg Edinburgh 1991; 36: 70 - 4.

Greenburg AG, McClure DK, Penn NE. Personality traits of surgical house officers: Faculty and resident views. Surgery 1982; 92: 368 - 72.

Scheuneman AL, Pickelman J, Hesslein R, Freeark RJ. Neuropsychologic predictors of operative skill among general surgical residents. Surgery 1984; 96: 288 - 95.

Vickers MD, Reeve PE. Selection methods in medicine: a case for replacement surgery. J R Soc Med 1990; 83: 541 - 2.