19 posts / 0 new
Last post
mosul
mosul's picture
Do belts work?

I am looking for some published research into the effecacy of the belt grade system (If no-one knows of any then I am willing to do it myself). 

Specifically how do practitioners feel about their grade, those above and those below them? Do the belts help retain students or increase dropout? I would be equally interested in research into reasons for dropout from Martial Arts, a googlescholar search has not been overly hlepful.

Thank you.

Les Bubka
Les Bubka's picture

Hi Mosul

I'm not aware of any such studies. Few years back I got rid off the belt system in our club, for some student it was ok (mostly older), after consultation with my group we started to use belts again as younger members (18-28) are much more motivated to training. Having a goal is important to them.

Hope that this helps a bit.

Kind regards

Les

mosul
mosul's picture

Thanks for the response Les, I find it fascinating that you decided to drop them in the first place. What was your original thinking and was it your observations or the students comments that led you to re-instate them?

My questions run something like this...

How do belts work, you know making someone wear a symbol that they are less than perfect? 

Does it work as a sieve where the weakest performers quickly drop, leaving the strongest who are happy to progress within a structure that has already confirmed them as successful? Or does it provide a safe structure for all to progress equally?

Unless you attend the Cobra Kai Club from the film Karate Kid, you probably subscribe to the later view that it is a meritocracy. I know that grouping students by ability is almost 'ilegal' in schools these days, so what is it about the belts or martial arts or both that makes it work?

 Mark

Marc
Marc's picture

mosul wrote:

I am looking for some published research into the effecacy of the belt grade system. [...] Specifically how do practitioners feel about their grade, those above and those below them? Do the belts help retain students or increase dropout? I would be equally interested in research into reasons for dropout from Martial Arts, a googlescholar search has not been overly hlepful.

Hi Mosul, I am not aware of any scientific studies regarding the drop-out rate of martial arts students depending on the use of coloured belts or ranks - or any studies regarding belts or ranks.

I'm pretty sure most of the members of this forum can provide valuable insight and anecdotal evidence, if you like. However, your question was about published studies.

mosul wrote:

(If no-one knows of any then I am willing to do it myself).

Yes, would be interesting to know.

Some thought on conducting the research:

First you'd have to formulate the exact question you want answered. (Else you might get "42" as an answer and not know what to do with it ;-) ).

Next you'd have to formulate a hypothesis that precisely sets the rules and constraints of an experiment that can answer the question.

Then you would need a test group and a control group who both train in comparable training environments. A double blind study would be ideal, but difficult to conduct within the given context.

You would need enough participants to have a statistically relevant sample size.

Finally you would collect and analyse the data and compensate for other factors that might have an influence on the drop-out rate.

I think it will be extremely difficult to find two comparable training environments, one of which uses belts/ranks and the other one does not. Plus, there will probably be so many other factors that can have an effect on the outcome that it will not be easy to compensate for them.

Anyway, I would love to read the study. Please tell us if you could need any help.

All the best,

Marc

Marc
Marc's picture

mosul wrote:

so what is it about the belts or martial arts or both that makes it work?  

A while ago I wrote an article about the use of belt colours in karate. It might be of interest to you.

All the best,

Marc

mosul
mosul's picture

Marc, Great article I enjoyed it.

I specifically noted the second paragraph in the Pyschology section, where you discuss the difference between aiming for the next belt v concentrating on technical weaknesses. I believe this succinct and insightful passage captures one reason that people felt MMA was necessary; that the belts had become a system of producing black belts rather than producing effective fighters. I have no evidence for this opinion.

The re-balance of this has been the rise of 'reality-based' self-protection i.e putting viable application back into the curriculum, or, when it goes south in the bar does your belt count for #@$%. 

All of which raises the point that the belt system is used irrespective of the curriclum or philosophy. It appears to work for sport karate, self protection, kids karate etc etc. It has worked for decades in every country on the globe. As a teaching tool it appears to provide learners with a clear structure of achievable yet increasingly challenging goals. Ones that keep them motivated to learn.

On an annecdotal level I can't remember anyone who dropped out of MA claiming it was because they realised they were incapable of getting the next belt. To lazy, bored, distracted, lack of time etc yes but not incapable.

So which of the following best describes the belt system?

1. A meritocratic pedagogy that promotes skill mastery through a structure that moves seemlesly from extrinsic to intrinsic motivation.

2. A soft gate keeper that turns up the pressure over time to winnow the dedicated from the diletantes.

3. A hard gate keeper that immediately demonstates to beginners that the road is long so best get out early.   

I think your point about research design is well made and needs to be born in mind by myself and anyone who is interested in following me down the rabbit hole. I also agree that the members of this forum probably represent a strong popluation of inquisitive practitioners who might want to know.

Thanks for your input.

Mark

Iain Abernethy
Iain Abernethy's picture

As has been mentioned previously, I doubt there is any proper research on this. Maybe “market research” from the more commercially efficient organisations, but I doubt there is anything that qualifies as a proper study.

It’s also problematic because there is no uniformity in the way the grades are used. They are used differently in different groups.

mosul wrote:
So which of the following best describes the belt system?

1. A meritocratic pedagogy that promotes skill mastery through a structure that moves seemlesly from extrinsic to intrinsic motivation.

2. A soft gate keeper that turns up the pressure over time to winnow the dedicated from the diletantes.

3. A hard gate keeper that immediately demonstates to beginners that the road is long so best get out early.

It can be any of them depending on the group, the system, and the individual. It’s therefore really hard to be objective across the board with a firm answer.

mosul wrote:
On an annecdotal level I can't remember anyone who dropped out of MA claiming it was because they realised they were incapable of getting the next belt. To lazy, bored, distracted, lack of time etc yes but not incapable.

I’ve had that. It’s not that uncommon. Some accept there are legitimate limits (in the same way not everyone who plays basketball can turn pro), and others feel the system is “unfair” for not dropping to their level.

Karate should be open to everyone, but not everyone can achieve the higher levels. Why should we think otherwise? Not everyone can run a mile in under 4 mins. Not everyone can bench-press their bodyweight. Not everyone can earn a doctorate. All of us have a unique set of gifts and limitations. It’s therefore should be obvious that there may come a point where a given individual can’t progress up the grading structure any further. Not all white belts are potential grandmasters.

People can and should keep on practising for the sheer enjoyment of it. I like reading about history for its own sake, but I know I’ll never have a qualification in it because I’m not particularly “academic”. If the fact I’d not get a degree in history made me quit reading about it, then it’s clear I never really had any passion for the subject in the first place. Same with martial arts, it should be acceptable to study and enjoy it without the need for higher dan grades to be open to all. Sadly, I have seen people quit because of this (and switch to groups with lower standards or no standards at all to “get” the previously elusive grade).

Always an interesting topic this and one with many strands and viewpoints!

All the best,

Iain

Les Bubka
Les Bubka's picture

HI Mark

mosul wrote:

 I find it fascinating that you decided to drop them in the first place. What was your original thinking and was it your observations or the students comments that led you to re-instate them?

My teacher always told us that belts are not important, they are by product of the training, also I have train in wrestling and there you have no belts or grades, just results. Looking around the martial arts scene I see that belts are no more representation of the skills. At that time I had a training group, most of the guys was experienced fighters or practitioners (most black belts) and we just wanted to train there was no need for belts, but due to Work, family and other reasons this group fell apart(shame). Now I have guys who start from scratch, they are young, new generation seem to be fuelled by goal and deadlines and they perform better when they have program to be finished within half year. At the moment in my dojo we have grading every 6 months for lower grades up to 5th Kyu, and a once a year for 5th to 1Kyu. I do encourage students to not look on the belts just train the best as they can, as Iain said not everyone should be Master. 

My opinion about black belt in the article below.

https://shinaido.wordpress.com/2014/07/19/black-belt-target-or-side-effe...

Kind regard

Les

 

mosul
mosul's picture

Marc, if you are still with us, It appears I might have to pursue this myself, you seem to have a thorough grasp of research design, if you have the time or the interest I would welcome your thoughts.

Les, thanks for more detail, and the blog on belts is very informative.

Your blog comment on being happy to have your black belt but realising it was not important is interesting. As is the implied change of motivational focus from the 14y old to the Dan grade. It appears that the belt system might have worked perfectly for you; broken the journey into achievable steps while allowing you to develop a deep love / knowledge of the art.

How do you feel about the new students focus on grading, is it something that you think they will 'grow out of' or is it your intention to use this focus as a neccassary evild in order to draw them into deeper learning?

Thanks.

Mark

Les Bubka
Les Bubka's picture

Mosul

It is very individual thing, some train for achievements other just for training, in my club I don’t force people to grade I'm happy whatever they choose.

Yes for me it was a drive when I was young as I had very low self-esteem and anxiety. Belts helped me to build on confidence and in a way fight with anxiety, after every grading I felt more confident and had more belief in achieving my goals. Later that changed when I started to study more in depth, now other things in karate motivate me.

mosul
mosul's picture

Iain thank you for your response in attempting to reply I have realised that there may be a very important distinction to be made here.

I am mainly talking about the kyu grade belt system. I am concernd with its effectiveness as a tool to allow all comers a clear path from ignorance to competency (which is my definition of first dan, please improve it).

As such are your examples of people leaving MA more about the higher dan end of things than the kyu grade end?

Whilst I accept the real MA is going on at first dan upwards, I wish to consider where the next generation of dan grades are coming from and how effective we are at performing the alchemy of turning white into black.

Mark

Iain Abernethy
Iain Abernethy's picture

mosul wrote:
As such are your examples of people leaving MA more about the higher dan end of things than the kyu grade end?

Mainly but not exclusively. I think most people (not all), if they apply themselves, are capable of achieving 1st dan. Typically, in my dojo, that takes the best part of a decade. I realise that’s much longer than in most, but we like what we do and have no desire to “drop” to what is the norm. It’s therefore hard to compare because in the same time period my group would be working for upper kyu grades, whereas in another group they may be working for 3rd dan.

There’s no uniformity in grading requirements and levels – which I think is a good thing because the only way you’d get that is enforced mediocrity – and hence grades are internal markers only.

You can compare grades within groups, but not across groups.

So there are definitely going to be people, kyu grades and dan grades, who will quit because they are not able to progress. If we train for the love of training, then that won’t happen. And it’s only those who love training that will work long enough and hard enough to fully capitalise upon their potential to reach higher dan grades anyway.

I did a podcast on the grading system a few years ago which may be of interest to some readers of this thread.

https://iainabernethy.co.uk/content/black-belt-and-grading-system

All the best,

Iain

Iain Abernethy
Iain Abernethy's picture

Tau wrote:
Sorry to digress from the topic, but Iain I've never disagreed with you so much over anything …

The way that you present material in a logical, ordered fashion with references and formulated arguments I think makes you one of the most "academic" people I've ever come across. Depending on the subject and course construction you could get a degree with relative ease. Your mind is clearly wired that way.

Thank you for the kind words … but oh, how wrong you are ;-) What I am good at doing is presenting MY thoughts in an ordered and logical way. It’s entirely different when it comes to the thoughts of others. As Becky accurately observed, “You are very good structuring a logical argument … so long as it’s your own argument”.

I do like studying all kinds of things, but I need to follow my own path through it. When given a “academic / formal path” through any topic, I normally quickly lose interest. If I’m honest, I also tend to rebel and push back when given constraints of any kind (which has been both a blessing and a curse). Formal learning to a set timetable is therefore something I’m not wired to do.

When studying something I am deeply interested in, I am good at wrestling into down to the core components. This does help me teach and structure things in a logical way, and I also think it helps me teach with enthusiasm and passion. The rejection of constraints on my thinking, the refusal to engage in an enforced direction of study, and the inability to accept conventional wisdom also means I have no “sacred cows” and hence I can see things afresh. It is these things that have helped me do what I do … and they are antithetical to academia.

In my family I am an the only one of my generation that left school at 16 and I’m the only one without a degree. And this outcome was obvious to all from a very early age. I’m just not wired that way … as those close to me will attest.

Tau wrote:
one quick view is that the grading system is both [the best] and worst aspect of martial arts. And we can flesh out both perspectives as much as we like.

I’d agree with that. They are internal markers only and are best viewed as such.

All the best,

Iain

mosul
mosul's picture

Iain,

Thank you for your reply. To chime in on the history issue, you know yourself best and clearly have an idea of what a degree would require. As in MA there are many forms of learning and speaking as someone who feels he has benefited enormously from yours I would suggest that efficacy not grade is what counts.

Can you see what I did there, can you, can you? 

I vaguely remember the podcast, but will check it out again.

I am going to push forward and hand out a few questionaires to see if I can get an idea of how students feel about the belt system. I will try to make it a 'good' study, so it will probably take a while. If I manage to complete it I will post back any results, just in case anyone is interested.

Thank you to all who have responded and helped me get a clearer understanding on what it is I am getting at.

Mark

Iain Abernethy
Iain Abernethy's picture

mosul wrote:
Thank you for your reply. To chime in on the history issue, you know yourself best and clearly have an idea of what a degree would require. As in MA there are many forms of learning and speaking as someone who feels he has benefited enormously from yours I would suggest that efficacy not grade is what counts.

Can you see what I did there, can you, can you?

Very good :-)

mosul wrote:
I am going to push forward and hand out a few questionaires to see if I can get an idea of how students feel about the belt system. I will try to make it a 'good' study, so it will probably take a while. If I manage to complete it I will post back any results, just in case anyone is interested.

Anything I can do to help, please just let me know. I’m sure others will also be keen to help out and see where it goes.

All the best,

Iain

Marc
Marc's picture

mosul wrote:

Marc, if you are still with us, It appears I might have to pursue this myself, you seem to have a thorough grasp of research design, if you have the time or the interest I would welcome your thoughts.

Hi Mosul,

I really wouldn't claim to have a thorough grasp of research design. I merely pointed out some steps you would have to consider if you would like to approach it scientifically.

For details of statistical methods and questionnaire design (if necessary) you would have to ask experts like statisticians, psychologists and the like.

Anyhow, why don't we start with the first point from my (non-exhaustive) list:

1) Formulate the exact question you want answered. (Be as specific as possible!)

mosul wrote:

a) Do belts work?

b) [What is] the effecacy of the belt grade system[?]

c) How do practitioners feel about their grade, those above and those below them?

d) Do the belts help retain students or increase dropout?

The enumeration in the quote is mine. So I can comment on the questions:

a) Too vague. - Work in what repect?

b) Too vague. - Efficacy with respect to what?

c) A totally different question. Still, too vague.

d) Another different question. Already fairly specific, and seems measureable. Still a little vague. What do you mean by "belts"? Is it belts as opposed to buttons (just kidding), is it belt colours, is it gradings, is it examinations, is it specific curriculae, is it setting specific training goals? Something else?

Let's try to find out what it really is you want answered.

Marc
Marc's picture

mosul wrote:

Marc, Great article I enjoyed it.

Thanks, glad you liked it.

mosul wrote:

On an annecdotal level I can't remember anyone who dropped out of MA claiming it was because they realised they were incapable of getting the next belt. To lazy, bored, distracted, lack of time etc yes but not incapable.

I'd agree. Whenever I asked people who stopped practicing karate why they would leave, belts have never been mentioned.

mosul wrote:

So which of the following best describes the belt system?

1. A meritocratic pedagogy that promotes skill mastery through a structure that moves seemlesly from extrinsic to intrinsic motivation.

2. A soft gate keeper that turns up the pressure over time to winnow the dedicated from the diletantes.

3. A hard gate keeper that immediately demonstates to beginners that the road is long so best get out early.

Depending on the dojo or association it could be any of the above.

However, I think the belt system does serve (at least) four purposes.

1) It helps the instructor when they want to give more complex tasks to experienced students and less complex tasks to beginners. The belt colours facilitate dividing the class into smaller groups.

2) It means that some instructor/examiner recognised your progress in learning your art. In this respect belt colours might not be necessary, because certificates or a ceremony might work just as well.

3) It can be a motivator for students as the belt colours represent defined milestones in their journey of learning their chosen art.

4) It provides students/adepts with a constant reminder of what they have achieved and at the same time of a certain responsibility that comes with the level of knowledge they have mastered so far.

I think it is important to note that ranks (and therefore belts) have meaning only within a certain dojo, group or association. They are not really comparable across the world.

MCM180
MCM180's picture

Sorry I'm a few days late on this.

I have some experience with research design - I've got a PhD in Accounting (which at the academic level is a social science much like economics) and have done a few studies in peer-reviewed journals.

I agree with Marc's general outline near the beginning of the post. I'd also encourage you to think carefully about what kind of answers you want. Do you want quantitative answers (e.g., belt systems increase motivation and participation by X%, or Y% of students say they use belts for motivation, Y% say they use them for feedback, etc.)?

Or do you want more qualitative answers (e.g., students gave these six reasons for belts being effective and these 4 reasons why belts are harmful; students of Eastern-style martial arts or those looking primarily for fitness benefits were more likely to find them helpful than students of reality-based or self-defense styles, that kind of thing)?

This is important because you have to carefully consider how to measure (that is, put a number onto) the responses you get for quantitative results. For example, if you want to determine whether belts increase "karate progress," you need to have a way to measure progress in karate skills OTHER than measuring belt ranks. Partly because some of your students will have to be non-belt students for the study design, and partly because OF COURSE students in belt systems will make progress as measured by new belts, so that's not a good measure of "karate progress" for your study. 

You may also want to consider measuring personality traits. Some people are very goal-oriented or motivated by extrinsic factors; others are more intrinsically motivated (as Iain describes himself) or process-oriented. There are validated measures of these personality characteristics in the psychology and social psychology literature that might help you capture how individual differences might matter. This could be something that helps instructors - maybe they could give personality assessments to students and help customize training & rewards to match individual students' desires.

Also: you'll also want to find students who dropped out of training altogether and those who've switched dojos, to get a sense of why they left. Was it belts, or the lack of belts, or was it other factors (time, money, injury, etc.), or something haflway in between (lack of interest, which may or may not be related to belts)? If you only survey students who are continuing, then you get a biased sample. So you'll want to ask your instructors for permission to contact some of their former students. And you'll want those students to be sure that they can be honest with you and any negative comments about individual instructors won't be passed along. Otherwise your answers won't be very honest.

Finally, most of the people on this forum are pretty interested karate and a good number are instructors. So you might want to start your study by spending more time talking to casual students than to committed karateka. You'll probably get different insights. I know that as an accounting professor, I'm VERY different from the average accounting student (and even the average practicing accountant) who doesn't love accounting as much as I do. Instructors might not get good, honest answers from their own current & former students, so instructors' understandings of why students dropped out might be incomplete.

I'm also happy to help you design the study if you want.

Yours,

Christian

Marc
Marc's picture

Great post Christian, thanks for your contribution.