People who work in access services will be very familiar with the quest for quality. We all love quality. In the UK, Ofcom is very keen on measures to improve it. In Spain, a number of institutions have come together to decide what it is. Students the world over have been writing MA theses on it for years. And of course there are countless righteous tweets and side-splittingly humorous articles about errors in live subtitling, but who exactly gets to decide what quality is and how can we measure it?

I recently attended a meeting of a new working group in Spain with one of our clients. The group is tasked with deciding what the quality indicators and measurements are going to be for access services in Spain. It’s made up of broadcasters, disability lobby groups, regulators and government agencies and if my first meeting was anything to go by, they really have got their work cut out.

Deciding on measurable quality indicators for access services is no easy task. This group’s approach is to decide what indicates quality and then work out if it can be measured and how. We had relatively few problems agreeing on what was important (synchronisation, speed, accuracy, readability, spelling, grammar, omission, tone of voice…). But the age old issues raised their heads again (verbatim vs. edited captions, non-dubbed content in AD). For example, we spent over an hour discussing subtitling non-standard pronunciation, without conclusion. Most of the association representatives in the room felt very strongly that non-standard speech should be rendered exactly (phonetically) and that anything else was not equal access. The point being that if a politician says “expresso” instead of espresso, deaf people have the right to know. Unfortunately, rendering that pronunciation phonetically is not equal access either. First, we could be drawing a lot more attention to it than it would have for a hearing person. We could also cause confusion — is that a spelling mistake or is the President a bit of a hick? Or could we (god forbid) even cause MORE people to think it’s correct to ask for an “expresso”? Should we also draw attention to all those who can’t pronounce segue, or that pronounce rioja rio-ka instead of rio-ha? Personally, I’d string up everyone who says chorit-zo instead of choree-tho.

The answer, in this case, as for a lot of what constitutes subtitle quality, is that you can’t make hard and fast rules. What we need to transmit is any relevant information carried in the pronunciation, and this will depend on a lot of factors. Subtitlers need to use their judgement to decide. And where you can’t have hard and fast rules, it’s hard to have objective quality measurement.

And the real challenge will come when we come to decide on those measurements, and importantly, weighting. Many indicators may actually work against each other. For example, in live subtitling, what’s better, a subtitle with a mistake in that’s understandable, or a perfect subtitle with 20 seconds delay? I’d say the former, but maybe that’s just me. Depending on your language, culture, experience and degree and type of hearing loss, you may feel completely differently. It all boils down to working out what the majority of the audience considers better, and that’s where it gets really difficult, because “the audience” is made up of a wide range of people with very different needs, preferences and agendas.

So who gets to decide? Who are the experts? Professional subtitlers? Academics? User associations? Broadcasters? The answer, in Spain, at least, is that we will try to reach a consensus between a number of stakeholders.

Wish us luck.

Diana Sanchez, Head of Operations, Europe, Access Services