During
the recent World Barista Championships we saw Stephen Morrissey (2008
WBC Champion) discussing WBC judges' scoring with the comperes,
Nicholas Cho, James Hoffmann and Stephen Leighton. For me his
insights were fascinating. For example, he said (and I'm
paraphrasing, so forgive me if this isn't exactly correct) that the
ideal scoresheet as completed by a judge is one that should need
no further verbal feedback...
that for every score on the sheet, the judge should write a comment
explaining that score well enough for the competitor to understand
why that score was given. Upon hearing that I couldn't help feeling
that, however, that Stephen's is perhaps an idealistic view of the
transparency within WBC scoring. My own experience was extremely
rewarding but it left me with many unanswered questions and not
enough clarity of what I need to do differently in future
competitions. (“Buy a watch” was the advice jokingly given by one
judge, after I overran by 37 seconds.)
Stephen
is on the Board of World Coffee Events (WCE), the organisation that
runs and regulates the World Barista Championships - including
coordinating judging activities. I wanted to understand more from
Stephen's perspective, so I wrote to him. I'm sure he's a busy man
and can't reply to every email from random baristas around the world.
Nonetheless, I stuck my neck out because if you don't try then you
will never know. Surprisingly, he did read my email and
contacted two of his colleagues at WCE – Ellie Matuszak and Carl
Sara, who are the Co-Chairs of the committee that oversees all of the
competitions. The three of them decided to respond!
Thanks
so much for agreeing to this. For the benefit of readers could you
please explain who you are?
We're
chiming in on behalf of the WCE Competition Operations Committee
(COC). The COC is responsible for the work around this topic.
Great!
So... judging & scoresheets seem to be a sensitive subject for
competitors. What are your views on how well the current system
works?
At
present judges are evaluated based upon the Judge Competencies
framework. However, no secrets... the current status of WCE is a bit
unusual. The assessment tool, the Judges Calibration Workshop, was
built before standardized training toward it was created.
So
there is a gap in how judges are trained to attain the competencies
required to judge?
That
is why we are working on that training now. In fact, WCE is
currently developing a standardized judge training program to be in
use at the National Body level. We formed a specific subcommittee to
develop these modules, the Instructional Design Subcommittee or IDSC.
Ah
yes. I recall Instructional Design from the SCAA Instructor
Development Programme. In short, it's a model for building training
courses.
You
will recall that proper Analysis, Design and Development can make all
the difference in creating a successful training program or not (this
is called the “ADDIE” model of instructional design). No point in
doing any Implementation and Evaluation unless and until the AD&D
are fulfilled. This training program is huge. It is going to be about
10 separate modules, and based entirely on training toward the Judge
Competencies.
Sounds
good, but what does this actually mean? What will be different?
The
changes you should expect to see range from the obvious to the
less-so. First, judges will be trained better, meaning they will be
explicitly trained toward successfully meeting the Judge Competencies
(= better judging: more fair, more transparent, more accurate)
That's
a lot of training. So to clarify, would this new training only be
given to judges at the World Championships, and not at a National
level such as the UKBC?
No,
actually the opposite. We still will not be training judges at the
World Level- all training will take place at the National Body Level.
Judges can qualify to judge in the World competitions by passing the
Judge Certification Workshop. This means that judging from nation to
nation will be more consistent. Judging from the National Body level
to World Barista Championship level will be more similar. No more
surprises as in "my Head Judge trained me to do it a different
way" etc.
And
hopefully it will go all the way through to Regional judging too
(i.e. during National Heats), where there have been some high profile
critics of the standard of judging. How do you hope to get this
training delivered across so many judges in different places?
Long
term, we hope to continue to leverage the parent companies to grow
and distribute the training, but we are still working out the details
of what that means exactly. (Full disclosure, I am employed by one of
the parent companies.)
So
we should have better trained judges in future, who will all have a
better understanding of the rules and how to apply them. Great! But
what about competitors? We need a clear definition of what the judges
expectations are. No secrets! I have previously suggested a
Competitors Calibration Workshop. What is happening to address that?
One
of the forthcoming changes is that anyone who wishes to can sit the
National Body level judge training. Active barista competitors may
not judge in competitions, but can sit the training. Other
interested parties who want to learn about WBC (coaches, volunteers,
newbies) can participate in the training and learn something without
committing to judge (though we hope they do if they want to).
So
attending the judges training will help to prepare competitors for
their performance on competition day?
In
fact, some modules will be specifically designed for use in the
months prior to a competition, rather than at it.
Thanks,
Ellie. It sounds like a big improvement to me.
Thank
you for your candor and enthusiasm for the process of improving WCE
activity as an educational tool.
--------
As
a first time competitor in 2012 I saw many good things and some
not-so-good things. There is currently, I believe, an unnecessary
communication-gulf between competitors and judges. Some judges seem
to recognise this but are unsure how to make things better. When I
have previously asked for more clarity from judges, I have been told
“we
are not allowed to coach competitors”.
This problem presents a great opportunity for improvement! Both
parties could and should work together more collaboratively. It
seems to me that the guys at WCE have become aware of that, and are
being very proactive in addressing it in a professional manner. I
think the changes mentioned above are nothing short of a giant leap
for barista-kind, and I look forward to seeing how this is rolled out
globally. There will be challenges involved in implementing the new
training program, I suspect, but if it is implemented well then the
outcome will be a larger number of highly skilled baristas throughout
the world due to the competition being an even better learning tool
than it currently is. In my view the WBC remains one of the best
possible avenues for learning and fulfilling our potential as
baristas, and the future of the competition just got brighter.