What is AQuA?
AQuA is a copilot tool developed by SIL International that seeks to develop capacity and increase the thoroughness of translation quality assurance. It harnesses artificial intelligence (AI) techniques, assisting translation teams in objectively assessing multiple facets of translation quality by enhancing and complementing preexisting translation work. AQuA runs several augmented quality assessment methods in the areas of accuracy, naturalness, and clarity, which are then analyzed by translation teams to help them make more informed decisions about the quality of their translations.
To this end, AQuA is creating tools for:
- translators, aiding them in identifying obvious problems so they can address them earlier in the translation process;
- church leaders and consultants, allowing them to do their checking work more thoroughly and more consistently;
- administrators and strategists, helping them to compare and evaluate different methodologies and products (including new AI-based translation methodologies proposed by TBTA, SIL, and Avodah).
By bringing attention to possible anomalous passages in a draft, AQuA accelerates the quality assessment process by helping:
- translators make corrections (and produce an improved draft) prior to other quality checks;
- church leaders and consultants gain an “at-a-glance” understanding of quality across a draft and quickly dig into the relevant, granular quality issues; and
- administrators and strategists more easily track quality across projects to guide strategic allocations of checking resources.
Our hope is that AQuA would empower translation teams in the quality checking process, helping them to more easily spot the obvious errors, which in turn lets them focus more of their precious time and attention on the parts of a draft that really need their expertise.
AQuA as an AI Copilot
We are seeing a recent explosion of interest in AI-driven tools that are paired as “copilots” with human workers. These include copilots that are helping people to write code, draft emails, diagnose patients, write recipes, and much more.
The use of the term copilot in these instances emphasizes two important points:
- A person is in control of the process (i.e., they are the pilot)
- The person’s work is enhanced when it is assisted by the AI tool (in terms of efficiency, thoroughness, accuracy, creativity, etc)
In Bible translation, there are individuals such as church leaders, translators, and consultants, who undergo specialty training to create high-quality translation drafts. AQuA, an AI quality-checking copilot, is able to help these translation experts locate anomalies and issues within translation drafts relating to the areas of accuracy, naturalness, and clarity that would likely not have been found otherwise.
It is important to note that AQuA is not designed to give insight into issues related to the other important qualities of a draft, such as the beauty or acceptability of a translation. AQuA is a tool which helps stakeholders in the translation process to more efficiently and thoroughly evaluate certain qualities of a draft translation; with the help of AQuA, teams can improve several qualities of their draft within the amount of time allotted to them.
What does AQuA Measure?
The quality of a translation is not a singular entity; rather, it is best approached as a whole set of different sub-qualities. With AQuA, we are not attempting to come up with a singular estimate of quality, but only to assess a subset of these component qualities.
With AQuA, we do not assess these such-qualities directly; we only deal with qualities that can be measured algorithmically and expressed as a number. These measurable qualities are never an exact match to one of the abstract qualities (such as accuracy, naturalness, or clarity); instead they work like proxies for key aspects of the abstract qualities. For example, the AQuA metrics for Word Correspondence and Semantic Similarity are both quantifiable measures that relate to the accuracy of a translation, but do not give a full picture of its accuracy.
Methods of Assessment
A list of AQuA assessments are provided below, grouped under the abstract qualities of accuracy, naturalness, and clarity:
Accuracy
Word Correspondence
Semantic Similarity
Missing and Added Words
Naturalness
Sentence and Word Length (Lix Readability)
Clarity
Comprehensibility (Coming Soon)
https://www.youtube.com/watch?v=6MIDtmwJfZM
AQuA is currently available via a standalone web application, with plans to integrate into Clear Dashboard, Scripture Forge, and platform.bible. If your team is interested in using AQuA, please email us at idx_aqua@sil.org.