Case studies



the MTQ48 mental toughness assessment at a leading European business school

the Checkpoint 360 feedback appraisal system

the Profile XT (PXT) in financial services

the Profiles Sales Indicator (PSI) in retail

the Profile XT (PXT) in leisure (hotel industry)



the use of the MTQ48 at a leading European business school

Background
The business school, based in mainland Europe, is an elite business school which was ranked in the top three schools worldwide in its field.

The flagship programme was a Masters programme which attracted elite students from around the world. Graduates from the school were generally highly sought after and were easily placed into excellent jobs within largely blue chip businesses. Like a lot of business schools they had observed that within each semester intake (3 semesters per year) there was a significant and worrying drop out rate. 30% of students eitherfail to complete the programmedrop out to return at a later date or
changed course to an lower level programme.

This has a number of actual and potential implications. Foremost amongst these was the loss of income when the above happened. However equally concerning was student welfare and student satisfaction.

One thing about which the school was fairly adamant was that the students were the elite and should generally be up to the challenge of the demanding course.

The Study
When the organisation became aware of the Mental Toughness model and the associated measure – MTQ48, it decided to test the entire spring semester intake – around 125 people. The objective was simply to see whether the data provided any indication which might help to explain what was happening. The students were newly into the term (about 10 days) and completed the questionnaire in one supervised sitting. They used the on-line version and everyone completed it in English, the teaching language of the school. Most students weren’t native English speakers but all had English as a working language.
The results were fascinating.

The Results
The students were the elite in one sense. Their academic record to this point in their lives was second to none. However the Mental Toughness scores were not significantly different to those of the normal population. Given that this was pressured environment this suggested that a significant proportion (around 30%) would find the course tough – maybe too tough.
What emerged was a fascinating set of results.

Implications for each MT scale:
Control – How will they deal with a varied and complex workload? Time Management is a key issue here.
Challenge – How will they deal with challenge? The Masters programme is one big challenge.
Commitment - How will deal with working to clear and demanding goals? Dealing with a range of closely targeted events could be an issue.
Confidence - How will they deal with setbacks and major problems? Dealing with criticism and things not going according to plan

The study showed the following:
Overall assessment of Mental Toughness
The Overall Mental Toughness profile was significantly lower than expected. The distribution of scores was not significantly different from a normal distribution. These might be academically elite, they are as sensitive as most people on the whole. More than 30% had scores of sten 4 or lower

Challenge
When the Challenge scale was examined this too had a profile which was lower than expected. Most students had scores which were typical of the average person. Only a small proportion had scores which suggested they would respond well to challenge, change and variety. Again a significant number had low scores.

Commitment
Examining responses to the Commitment scale also showed that the majority of students (70%) had scores which were no different to average scores in the general population. Only a small proportion had high scores for commitment. This is particularly significant because the essence of a Masters programme is the focus on achieving a very specific measurable goal in a finite time scale.

Control
Analysis of the scores for Control showed that the curve was skewed to the left. This is perhaps the most interesting aspect of the pattern of scores. One implication of control is that it determines to what extent the individual will deal with a multiplicity of tasks and will plan to deal with those effectively. Being given a variety of task and activities and to be expected to prioritise and deal with these in fairly short time frames will create problems for most of this population. If the students had arrived at this school from establishments which “spoon fed” education then the students may not have developed this capability to multi-task. What Seligman calls “learned helplessness”.

Confidence
Finally examining scores for Confidence was also revealing. Less than 25% of students had high scores on the confidence scale. Confidence can be considered a driver for Mental Toughness – it reflects the individuals ability to deal with set backs and keep going even in the face of difficulty. The Masters programme was certainly tough and challenging and had lots of scope for setbacks. A surprisingly large proportion of students had low scores (stens 1 – 3) If people had been exposed to long periods of pressure, stress and challenge, they can get worn down and their Confidence suffers.

OUTCOMES:
There was a strong correlation between Mental Toughness scores (MTQ48) and student drop-out rates. Mental Toughness is developable in most individuals and targeted interventions should lead to improved Mental Toughness and lower drop out rates.

Pastoral care could and should be more pro-active. Screening each intake would identify people most at risk. Interventions should be directed towards them. This does not require additional resources. It uses existing resources more effectively. Supporting students better should lead to higher student satisfaction scores. There is a compelling economic case for doing this. To test 125 students would cost around £1500. There are few additional costs for the interventions. Each student lost, loses between £6000 and £10000 of income to the academic institution.
“Rescuing” one student provides an immediate positive pay back.

[top]

 

Checkpoint 360º feedback appraisal produces significant results
When Bibby Financial Services reviewed its strategic vision in 2005, it set itself a challenging two-fold objective of outgrowing its core market in the UK and growing its business internationally. In order to achieve this, the business recognised that it needed to focus on its people who were expected to turn the company’s vision into reality.
Prior to launching this significant expansion programme, Bibby Financial Services had to ensure that:
- the right people were in the right roles
- the company had implemented a personal development programme for all staff


The Solution
After examining Bibby Financial Services’ management development needs in 2005, Miriam Koller, the Group HR Director, realised that an important part of the requirements package was a world-class 360º feedback tool. This would help her identify:
- the current strengths and development needs of the management team
- the priorities for developing a world class team

An initial market assessment was undertaken in order to identify the best tool to satisfy their demanding criteria. Following an exhaustive selection process, Bibby chose Profiles International’s Checkpoint 360, a highly sophisticated feedback tool, used in small, medium and large enterprises globally.


Why Checkpoint?
As Miriam Koller explains: ‘Checkpoint is very well presented. In a time-pressured environment, it is very visual, very easy to understand and explain because it goes through a series of building blocks. Rather than bombard people with lots of written information, it’s clear and graphic aiding fast interpretation of performance.It also makes it particularly easy for people to gain feedback about their working relationship with not only their boss but also their direct reports and peers.
This is the beauty of all of the Profiles products. They have been developed by business people for business people to be used in a business environment.'


The Results
In 2008 Bibby Financial Services employs 800 people in 34 locations around the world, and business growth has been excellent. In fact, in 2007 Bibby grew 16% year on year against a market rate of only 7%.

Koller and her Board colleagues are clear about what has led to this exceptional growth. ‘Initially,’ she says, ‘it’s about getting the right people in the right jobs and then allowing them to develop in their roles.’

She continues: ‘Without a doubt, Checkpoint has contributed to people’s improvement. It has held a mirror up to people. It has provided objectivity in terms of comparisons, trends and specific issues. And because most people like attention, it makes them feel good and realise that we want to give them time, develop their competencies and make them better managers.’

‘As a business we have started to do more individual coaching programmes rather than one-size-fits-all Executive Development programmes. We pick areas to focus on with different people at different times. We are growing at a fast pace and finding skilled people in the marketplace is tough so we need to focus more effort internally to expose our own hidden talent! Retention of our people is key, coupled with ensuring we help them achieve their maximum potential.

And the people who have used the assessments are equally convinced. Bibby Financial Services’ Group Marketing Director Diane Blinkhorn says: 'A real benefit was being able to compare the results this time round with the results from a Checkpoint completed a few years ago. You’re able to see the ground you’ve covered as well as perhaps identifying some new areas of focus as our roles grow and evolve.’

Koller explains: ‘The online element has been very important to us. Online seems more efficient than pieces of paper...I don’t think we could cope with any other system.

‘We have also used the multi-lingual element and over the next 12 months we will use it more within our overseas operations – in France, Poland, Czech Republic, Slovakia, India, Germany, Australia, Canada and the USA. One of the most important things we have learned is that you cannot overestimate what gets lost in translation, even if someone speaks excellent English.'

For the full text version of this case study, click here.

[top]



Assessment: the Profile XT (whole person assessment) in Financial Services

Objective: to determine a correlation between the Profile XT and employee turnover

Dept: member services
No. staff assessed: 56
No. staff in department: 60
Study conducted 2001/2002

Results
All the scores across each scale were analysed and compared to actual labour turnover. Cognitive ability (the Learning Index) is highly predictive in terms of job matching and whether people will stay or leave so this was examined closely. Participants in the sample who scored less than 6 had a turnover percentage of 64%. Participants who scored greater than or equal to 7 had a turnover percentage of 14%.

The Learning Index is made up of verbal and numeric reasoning. Verbal indices are made up of Verbal Skill and Verbal Reasoning.
Participants with a Verbal Skill score of 7 or less had a turnover percentage of 59%.
Participants with a Verbal Skill score of 8 or more had a turnover percentage of 33%.

Participants with a Verbal Reasoning score of 5 had a turnover percentage of 82%. Participants with a Verbal Reasoning score of 6 had a turnover percentage of 10%.

Participants with a Numeric Ability score of 6 or less had a turnover percentage of 75%.
Participants with a Numeric Ability score of 7 or more had a turnover percentage of 12%.

Participants with a Numeric Reasoning score of 7 or less had a turnover percentage of 57%.
Participants with a Numeric Reasoning score of 8 or more had a turnover percentage of 29%.

With regard to Occupational Interests those participants who scored between 6 and 8 had a labour turnover of 27% whereas those who scored at 9 and above had a turnover of 75% and those who scored at 5 or less had a turnover of 66%.

Benefits
Lower labour turnover from 80% to 18% saving £222,000.
Average cost of hiring £6000
At 80% turnover (60 x 80%) = 48 x 6000 = £288,000
At 18% turnover (60 x 18%) = 11 x 6000 = £66,000
Saving £222,000

Recommendations
The following scales for assessment changed:
Verbal Skill from 7-9 to 8-10
Verbal Reasoning from 5-8 to 6-8
Numeric Reasoning from 5-8 to 6-8
Financial/Administrative from 7-9 to 6-8
Energy Level from 4-7 to 7-9
Assertiveness scale from 3-5 to 2-4
Find candidates that are 80%+ match on the Learning Index
Find candidates that are 75%+ match on the overall job pattern

[top]



Assessment: the Profiles Sales Indicator (PSI) in Retail

Objective: to assess the impact on both labour and revenue turnover using the PSI

Dept: sales - account managers
No. staff assessed: 370
Number of top performers: 12
Number of bottom performers: 12
Study conducted 2001/2002

Results
2001 labour turnover rate: 65.5% (of the 84 hired, 55 left)
2002 labour turnover rate (using the PSI): 23.3% (of the 86 hired, 20 left)
Top performers in 2002 averaged a match of 87.7% to the job match profile
Bottom performers in 2002 averaged a match of 76.7% to the job match profile

Benefits
Reducing labour turnover from 65.5% to 23.3% had a major cost saving benefit.
Average cost of hiring £7500 (conservative)
In 2001 55 left. At £7500 each this is a cost of £412,500
In 2002 only 20 left. At £7500 each this is cost of £150,000
Saving £262,500

Through the job match profile (pattern), the company can identify top performers. Replacing bottom performers with top performers can have a dramatic effect.
Average sales of bottom performers £6220 (10.8% of target)
Average sales of top performers £95850 (133.6% of target)
Replacing a top performer with a bottom performer of which there were 12 would result in an increase in revenue turnover of £89630 x 12 = £1.08m!

Recommendations
A review of the project did however identify critical areas where information was either not available or incomplete. The following recommendations were made:
- ensure consistency (assess all relevant staff using the Profile XT)
- gather additional staff data to identify performance trends
- rank performance data
- refine job match profile (pattern) using additional staff data
- schedule a 90 day review once the data has been collected

[top]


Assessment: the Profile XT (whole person assessment) in the Leisure (Hotel) Industry

Objective: to improve international sales using the Profile XT where most sales executives were not meeting their targets

Dept: sales
No. staff assessed: 150
No. staff in department: 133
Study conducted 2006/2007

Results
Top Performers averaged 80.5% to the initial job match profile (pattern).
Poorest performers averaged 72.0% to the initial job match.

The following year, with the information gained from the first series of assessments the job match profile (pattern) was based on the top peformers with small adjustments made for certain personality variations. However only the 2 of the top performers were assessed along with only two of the bottom performers. The results were as follows:
Top peformers 95% to the new job match pattern
Poorest performers 73% to the new job match pattern

Benefits
Average sales per sales executive: £213,450
Average sales of top performers: £322,560 (7 top performers)
Average sales of bottom performers £90,690 (7 bottom performers)
For every bottom peformer replaced by a top performer will produce an increase in sales of
£231,870

Recommendations
- To revisit the job profiles (patterns) in 6 months to further fine tune them. This will become easier as more experience is built up.
- Continue to assess all staff regularly to improve sales and further develop (improve) the job pattern.