[2011-07-05 – This document is reproduced with permission of the Localization Industry Standards Association. All emendations to the original text are indicated by striking through the original text and inserting new text in red bold face in square brackets.]
[Note: This document introduces no technical changes to the GMX-V Standard with respect to the versions published on the LISA Website. All changes relate to locations of files on the Internet and to copyright licences.]
[The following changes to the copyright and license agreement are made with the permission of the LISA Board and Director as part of dissolution proceedings for the Localization Industry Standards Association.
This document is made available by the Localization Industry Standards Association [LISA] under the terms of the Creative Commons Attribution 3.0 Unported (CC BY 3.0) license:
Visit http://creativecommons.org/licenses/by/3.0/ for full license details.
You are free:
Under the following conditions:
With the understanding that:
Copyright © The Localization Industry Standards Association [LISA] 2007. All Rights Reserved.
This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to LISA.
The limited permissions granted above are perpetual and will not be revoked by LISA or its successors or assigns.
[All inquiries concerning rights not addressed in the emended license agreement above should be sent to Mr. Micheal Anobile at micheal_anobile@yahoo.com.]
This document defines the LISA Global Information Management Metrics eXchange Volume (GMX-V) specification. The purpose of this vocabulary is to define the metrics that allow for the unambiguous sizing of a given Global Information Management task. GMX-V is one of the tripartite Global Information Management standards which encompass volume (GMX-V), complexity (GMX-C) and quality (GMX-Q).
This document constitutes an initial draft for discussion.
This document and the information contained herein is provided on an "AS IS" basis and LISA DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Appendices
GMX-V addresses the issue of quantifying the workload for a given localization or translation task. This is often commonly referred to as word counts. Word counts, however, do not convey the true range of possible metrics that can be used to assess the cost of localizing a document such as the number of screen shots for a software localization project, or page counts for a document layout task. GMX-V is a more precise definition of the metrics required for billing and sizing purposes.
In defining GMX-V, care has been taken to provide a definitive and unambiguous definition that also offers the widest possible scope for achieving an adequate description of the task load for a given Global Information Management project.
Metrics fall into two categories: directly verifiable from the file contents (words and characters) and unverifiable (pages, file units, lines, etc.). Some metrics may fall into both categories, depending on the circumstances. For instance, page counts may be verifiable from the file contents under some circumstances; however, under other circumstances, only a printout for a given page format can provide the basis for a count.
GMX-V does not preclude the existence of unverifiable counts, but is concerned with defining precise rules for verifiable counts based on the file unit that is being counted.
To this end, it is proposed that GMX-V cover both word and character counts, as well as allowing for other relevant count categories that cannot be verified electronically. Character counts convey the most precise definition of a translation task, whereas word counts are the most commonly used metric in the translation industry. GMX-V encompasses both measurements, thus affording both translation suppliers and customers with a choice as to which measurement most adequately reflects the translation task in question as well as allowing for other relevant metrics.
From the implementation point of view GMX-V is designed to co-exist as a namespace within other XML documents. The main target will primarily be XLIFF and Translation Web Services WSDL compliant documents, but any XML document that allows namespace extension points could host GMX-V namespace.
GMX-V has therefore the following aims:
GMX-V defines a precise mechanism for word and character counts irrespective of the context within which the standard is being used. It can therefore be used for the unambiguous definition of word and character counts for any electronic document. Section 2.14 defines aspects of the GMX-V specification that are specific to localization tasks.
GMX-V makes no assumptions about the format of the input document. Although it is envisaged that GMX-V will be applied predominantly to XLIFF documents, it is not limited to XLIFF or XML based documents. Any electronic document can be counted using GMX-V as long as the text is converted into the required canonical form described in Section 2.2. Canonical Form.
The following concepts are fundamental to the GMX-V specification:
A document is made of a number of text units. A text unit is either a stand alone piece of text within a document, or a subdivision of a stand alone piece of text into recognizable segments. Document metrics will be based on an accumulation of the word and character statistics of the individual text units. Any segmentation should be detailed in an SRX (Segmentation Rules eXchange format) compliant document. A separate count of text units can be maintained within the GMX-V specification (see 3.7. Text Unit Counts).
Where the metrics are based on an XLIFF 1.2 file the <source> child of a <trans-unit> element constitutes a text unit regardless of any additional segmentation information that may be available as per section '2.9 Segmentation' of the XLIFF 1.2 specification. The <source> element should always form the basis of verifiable metrics such as word and character counts in an XLIFF file. The reason for this is to insure consistency with the XLIFF file structure and is also due to the fact that even where it is used <seg-source> data is not guaranteed to be comprehensive.
A precise canonical (base) form for individual text units is required to provide an accurate and unambiguous basis for conducting metrics. Native forms of the text are often encumbered with extraneous proprietary formatting codes, which make the production of unambiguous statistics difficult.
The XLIFF (XML Localization Interchange File Format) normalized XML form using Unicode encoding of the source language text shall be used as the basis for the canonical form for GMX-V. XLIFF is an OASIS standard. This is the format in which the text appears in the <source> element of an XLIFF file <trans-unit> element. For audit purposes an XLIFF file is required for GMX-V metrics. The GMX-V count is undertaken on the basis of this XLIFF document. The <source> element shall always be the basis of the verifiable word and character based counts.
Example:
<source>An example of the canonical form of a text unit.</source>
The canonical form does not contain any embedded formatting characters, such as those that exist in an XLIFF document extracted from a RTF file. Any such characters must be removed to produce the canonical form. In addition any formatting characters representing a space must be converted to the standard SPACE character (U+0020).
Original XLIFF <source> element with embedded RTF codes:
<source>The <bpt i="1" x="1">{\b </bpt>black<ept i="1">}</ept><bpt i="2" x="2">{\i </bpt>
cat<ept i="2">}</ept> eats.</source>
Canonical form:
<source>The <bpt/>black<ept/><bpt/> cat<ept/> eats.</source>
The GMX-V metrics for the above are:
words: 4, characters: 15, inline elements: 4, punctuation characters: 1, white space characters: 3
GMX-V does not mandate that the electronic form of the document being counted is in XLIFF. Nevertheless it is much easier to conduct GMX-V counts on an XLIFF file. Independent verification of the GMX-V metrics requires that an XLIFF version of the file is available.
GMX-V assumes that the text that is presented for counting contains only the text that has been deemed as relevant for the task. For a localization task this would be only the text that is required for localization purposes.
Where the metrics are based on an XLIFF 1.2 file and segmentation information is available as per section '2.9 Segmentation' of the XLIFF 1.2 specification then the segmented data should be used rather than the unsegmented contents of the XLIFF 1.2 <source> element.
Unicode Version 4.1.0 forms the fundamental basis for the XLIFF canonical form for character encoding and for establishing word boundaries. Apart from the ISO 10646 two Unicode Technical Reports are used for establishing the canonical form:
Unicode TR 29 establishes the word boundaries that allow for words and characters to be counted. TR 15 establishes the actual canonical form for Unicode characters themselves. Normalized Form C is the form mandated by W3C for XML documents and can normally be taken for granted during any conversion from non-Unicode encoding form to Unicode using industry standard programming libraries.
Word and character counts are governed by Unicode TR 29 Version 4.1.0 - Text Boundaries, Section 4 Word Boundaries, which in turn relies on the Unicode TR 29 Version 4.1.0 - Text Boundaries, Section 3 Grapheme Cluster Boundaries rules. This standard unambiguously defines words as opposed to stand alone punctuation, white space or enclosing punctuation characters. All word and character counts will be on the basis of the Unicode TR 29 Version 4.1.0 - Text Boundaries, Section 4 Word Boundaries.
A full definition of the application of Unicode TR 29 Word Boundaries to the GMX-V specification is provided in Section 2.8
Not all GMX-V metrics can be strictly defined or verified. Verifiable metrics can be defied for an electronic document in XLIFF canonical form. Non-verifiable metrics require a mechanism such as manual counting to establish its accuracy.
Non-verifiable metrics are not subordinate in any way to verifiable metrics; it is only that they cannot be proven on the basis of a given electronic document.
For word and character counts, the code for any inline elements (either empty or having content) within the canonical XLIFF representation will be treated as being totally transparent, that is, they are be treated as not being present. Inline elements will be counted separately. This is detailed in the section 2.11 Inline Element Counts below.
Example:
<source>In this <g id="g1">exa<x id="x1"/>mple</g> the in-line codes do not form
part of the word or character counts but are counted separately.</source>
would be counted as:
<source>In this example the in-line codes do not form
part of the word or character counts but are counted separately.</source>
The GMX-V metrics for the above are:
words: 20, characters: 91, inline elements: 3, punctuation characters: 1, white space characters: 19
In the canonical form, any inline XLIFF elements that signify a space or new line character must have the equiv-text attribute set to a single space character if the space character were otherwise not present in the canonical XLIFF form. If an inline element has spatial characteristics, then it is up to the program that is generating the XLIFF file to set the contents of the equiv-text attribute accordingly. If the inline element has content, then the space character must precede and follow the start and end tags of the element if the equiv-text attribute is used to denote spatial characteristics.
GMX-V is totally agnostic regarding how the XLIFF file is created and cannot imply anything regarding inline elements.
<source>The HTML break element<x id="x1" ctype="x-html-br" equiv-text=" "/>represented here by the in-line "x" element was
not preceded by a space in the original document.</source>
Sub flow text within place holder elements needs to always be preserved. Sub flow text always implies that it is preceded and followed by a white space character. If no white space is present then it must be inserted in the canonical form:
<source>Start<bpt id="2">code<sub>Text</sub></bpt>end<ept id="2">code</ept>.</source>
Canonical form:
<source>Start<bpt><sub> Text </sub></bpt><ept>end</ept>.</source>
The GMX-V metrics for the above are:
words: 3, characters: 12, inline elements: 6, punctuation characters: 1, white space characters: 2
A separate "Inline Element" count is maintained for inline elements.
The following list defines white space characters:
In the XLIFF canonical form white space characters are trimmed at the start and end of a text unit. Within a text unit multiple white space characters are reduced to a single space. The only exception is where the xml:space="preserve"
attribute has been set for an element. In this case no normalization of spaces will occur and all white space characters will be counted.
A separate count 'WhiteSpaceCharacterCount' will be maintained for Unicode white space. White space characters are not included in the main character count.
Example:
<source>This sentence has a word count of 14 words and 13 white space characters.</source>
Words form the basic unit for counting for the GMX-V specification. The character count is also based on identified words, with the exception of scripts that do not use space word separation. Word separation is described in this section
Words are defined according to Unicode TR 29 Version 4.1.0 - Text Boundaries, Section 4 Word Boundaries, which in turn relies on the Unicode TR 29 Version 4.1.0 - Text Boundaries, Section 3 Grapheme Cluster Boundaries rules. Unicode TR 29 Section 4 defines detailed Boundary Property Values and Boundary Rules which distinguish words from other grapheme clusters such as punctuation characters. These form an integral part of the GMX-V specification.
The following example, taken from Unicode TR 29, shows an example of the identification of grapheme boundaries:
| The | quick | ( | " | brown | " | ) | fox | can't | jump | 32.3 | feet | , | right | ? |
Followed by the extracted words:
| The | quick | brown | fox | can't | jump | 32.3 | feet | right |
In addition Unicode TR 29 Section 4 provides an optional rule for the apostrophe character which relates to French and Italian usage such as "l'objectif". This rule known as "Break between apostrophe and vowels (French, Italian)" must also be applied for GMX-V. Apostrophe includes U+0027 (') APOSTROPHE and U+2019 (’) RIGHT SINGLE QUOTATION MARK (curly apostrophe).
Thai, Lao, Khmer, Mynmar, Chinese, Japanese and Korean scripts are exempt from the word rules. See Section 2.13. for details of how these scripts are treated within the GMX-V standard.
Hyphen characters will not be treated as word break characters. Hyphens include U+002D HYPHEN-MINUS, U+2010 HYPHEN, U+058A ARMENIAN HYPHEN and U+30A0 KATAKANA-HIRAGANA DOUBLE HYPHEN, and will form part of the character count if they appear as part of a word as in 'Italian-American'.
No additional tailoring of the Unicode TR 29 Version 4.1.0 - Text Boundaries, Section 4 Word Boundaries rules is permitted in the GMX-V specification.
Example:
<source>This sentence has a word count of 9 words.</source> <source>This sentence/text unit has a word count of 11 words.</source>
The character count is predicated on the word count detailed in Section 2.8 above. For Thai, Lao, Khmer, Mynmar, Chinese, Japanese, Korean and other scripts that do not use spaces between words, the character counts are based on the non-punctuation grapheme boundaries. For all other scripts the character count is based on the identifiable words. Please refer to Section 2.8 above for a detailed explanation.
Characters are counted based on Unicode encoding according to Unicode TR 15 - using Unicode Normalization Form C.
An additional separate count is maintained for punctuation characters.
The following list defines what are to be considered Unicode punctuation characters:
The only exceptions are the 'hyphen' and 'apostrophe' characters if they appear within a word as in: can't, or out-of-the-box. These will be counted as normal characters and form part of the 'TotalCharacterCount' as detailed in Section 3.4. Character Count Categories
The apostrophe character count is qualified for French and Italian as per the Unicode TR 29 Version 4.1.0 - Text Boundaries, Section 3 Grapheme Cluster Boundaries rules "Break between apostrophe and vowels (French, Italian)" rule as described in section 2.8 above. In this rule the 'apostrophe' acts as a word break and is also counted as part of the character count.
Hyphens include U+002D HYPHEN-MINUS, U+2010 HYPHEN, U+058A ARMENIAN HYPHEN and U+30A0 KATAKANA-HIRAGANA DOUBLE HYPHEN, and will form part of the character count if they appear as part of a word as in 'Italian-American'. Please refer to Section 2.8 above for a detailed explanation.
A separate character count 'PunctuationCharacterCount' will be maintained for all punctuation characters in the document. Please refer to Section 3.4. Character Count Categories for details.
Inline elements give an indication of the complexity of the localization task. Among inline elements, a separate count will be maintained for elements that reference other elements. An additional count of inline elements will be maintained for each text unit's categorization category detailed below. Inline elements with content will be counted as two inline elements.
Example:
<source>In this <g id="g1">example</g>
the in-line codes do not form part of the word or character counts but constitute a
separate inline element count of 2, because the inline element has content.</source>
<source>In this <g id="g1">exa<x id="x1"/>mple</g>
the in-line codes do not form part of the word or character counts but constitute a
separate inline element count of 3, because we have one element with content and
one without.</source>
An additional count of inline elements that link to another text unit will be kept. Linked elements require additional localization effort as the linked text unit needs to be referenced as part of the translation. These elements are also counted as part of the inline element count above.
Example:
<source>In this <g id="g1" xid="t2">example</g> the in-line element references another trans-unit via the xid attribute -
it forms part of the inline element count as well as the linking inline element count.</source>
Scripts such as Thai, Lao, Khmer, Mynmar, Chinese, Japanese Korean and any other scripts that do not use spaces between words, are exempt from the word rules detailed in Section 2.8 above. For these languages no subdivision of grapheme boundaries into individual words will be accommodated in this version of the GMX-V specification. The Unicode TR 29 - Text Boundaries, Section 3 Grapheme Cluster Boundaries rules will still apply to distinguish text from punctuation characters for these scripts. These will be used to provide character counts for these scripts. Please refer to Section 2.8 and Section 2.9 above for a detailed explanation.
Apart from counts based on words, all other counts are relevant for these scripts.
The following count categories are specific to localization industry tasks.
A typical translatable document will contain a variety of types of text units. Some of these will require translation and some will not, while other text units will require only proofing since they have been matched against a leveraged translation memory database. Regardless of the agreement between a translation supplier and a customer, a count for overall text units, as well as translatable and non-translatable, will be provided for both word and character counts. In the following XLIFF based examples the canonical form is that of the <source> element.
Example:
<trans-unit id="t1" translate="yes">
<source>This is an example of translatable text</source>
<target>This is an example of translatable text</target>
</trans-unit>
<trans-unit id="t2" translate="no" state-qualifier="x-alphanumeric">
<source>10AB1024</source>
<target>10AB1024</target>
</trans-unit>
<trans-unit id="t3" translate="no" state-qualifier="x-punctuation">
<source>-</source>
<target>-</target>
</trans-unit>
<trans-unit id="t4" translate="yes">
<source>matched sentence</source>
<target state-qualifier="leveraged-tm">zdanie dopasowane</source>
</trans-unit>
Additionally, prior to sending the translation project to the translation supplier, the customer may have analyzed the source document against a translation memory in order to retrieve previously-translated segments. In this instance, an additional word count of the segments that are found in the translation memory may be provided
Example:
<trans-unit id="t1" translate="yes">
<source>This text unit has been matched against a leveraged matched database.</source>
<target state-qualifier="leveraged-tm">To zdanie zostalo dopasowane z bazy danych.</source>
</trans-unit>
Certain text units, such as numeric or measurement-only text units, may be converted automatically by software into the target language.
Example:
<trans-unit id="t1" translate="no" state-qualifier="x-numeric">
<source>10,000.00</source>
<target>10.000,00</target>
</trans-unit>
<trans-unit id="t2" translate="no" state-qualifier="x-measure">
<source>10.50 mm</source>
<target>10,50 mm</target>
</trans-unit>
It is up to the translation supplier and customer to agree on the exact nature and type of non-translatable text units. Text unit categorization will not be mandated, merely offered in the standard as an option.
Unqualified text units are text units that require translation. Any text unit that is not qualified as per Section 2.14.1 and has no exact or leveraged match is deemed to be unqualified.
This classification is important as any auto text (see Section 2.17. Auto Text) or inline linking (see Section 2.11. Inline Element Counts) and non-linking (see Section 2.12. Linking Inline Elements) counts are only applied to unqualified text units.
One of the main aspects of this standard is to produce an unambiguous industry accepted figure for the total quantity of words and characters within an electronic document. For translation tasks the minimum required by the specification, is that a volume metric is produced for words and characters that includes the following:
Over and above this minimum conformance for translation tasks (see 3.9. Conformance) required by the specification, it is up to the customer and supplier to decide how the other count categories will be applied to the translation task at hand. The specification provides a flexible and comprehensive vocabulary for customizing this calculation, but does not attempt to mandate any one given solution.
The built-in XML entity reference characters <, >, &, ", and ' will be counted as a single character for character counting purposes. It is a requirement of the GMX-V Canonical Form that all XML entity references are resolved prior to counting.
When counting XML documents user defined entity references may be encountered. The GMX-V Canonical Form canonical form requires that all user defined entities must be fully resolved prior to counting.
Within a document it is possible to identify text segments that can be handled automatically. Items such as numeric values, e.g. 10 or 10,000.00, measurement units e.g. 10.5 mm, standard phrases or acronyms e.g. WYSIWYG or trade names e.g. "Weapons of Mass DestructionTM".
It is possible to maintain word and character counts for such treatable text. This can be used to identify and charge for these categories at a different price. The categories will follow closely those defined in section 2.14.1 Qualitative Text Unit, although they will apply ONLY to 'unqualified' text units which do not have fuzzy matching.
Unqualified in this sense relates to text units that are not already covered by a qualitative category. This count category may be referred to as auto text for short.
Within a document the same unmatched text units may occur multiple times. This fact can be exploited by translation software to automatically populate subsequent repeating text units once the first one has been translated. Subsequent occurrences can therefore be automatically qualified as 'repeat' text in the same manner as leveraged matched text. Auto text metrics can therefore only be applied to the first occurrence of the text unit and not those qualified as 'repeat' text units.
Repetition counts should be based on the non-normalized form of the text unit in order to insure that the repetition is based on the exact form of the text. For example text units with identical text but different inline elements are not repetitions.
Current commercial practice varies from product to product. There is no unified method of providing an industry wide set of metrics. The GMX-V specification provides a level of detail which provides an adequate way of reconciling GMX-V with metrics provided by commercial practice which is based on accepted standards such as Unicode TR29-9, SRX and XLIFF.
The actual makeup of the count is up to the supplier and customer.
The count element type attribute contains the count category. The following count categories are provided for:
Auto Text comprises automatically identifiable text such as numeric values, date and trade marks or model names that can be automatically processed or ignored during translation. Inline elements relate to elements that occur within the text units that are being counted. Linking Inline Element Count Categories relate to linking elements that occur within the text units that are being counted such as HTML 'a' elements.
The count categories specified in this document are designed to accommodate the most frequent types of count encountered. While these cover the most commercial requirements it is envisaged that they will not be sufficient to cover all instances required. To this end an extension mechanism is provided. All extension count types will start with 'x-'. The following extension conventions apply:
'WordCount'.'CharacterCount'.'OtherCount'.'NonTranslatable' before the count type text that ends the count type.'Translatable' before the count type text that ends the count type.An example of the use of the extension conversions would be 'x-SomeDescriptionOtherNonTranslatableTextUnitWordCount'. These conventions are designed to facilitate the accumulation of appropriate count categories automatically.
The following count concepts are fundamental to the GMX-V standard:
The following word counts will be provided by symbolic name:
TotalWordCountProtectedWordCount'protected', or otherwise not translatable (XLIFF text enclosed in <mrk mtype="protected"> elements).ExactMatchedWordCountLeveragedMatchedWordCountRepetitionMatchedWordCountFuzzyMatchedWordCountAlphanumericOnlyTextUnitWordCountNumericOnlyTextUnitWordCountMeasurementOnlyTextUnitWordCountThe following is an example of the use of the extension mechanism for user defined word counts defined in the Section 3.1:
x-OtherNonTranslatableTextUnitWordCount'x-' and ends with the text NonTranslatableTextUnitWordCount.x-TranslatableTextUnitWordCount'x-' and end with the text TranslatableTextUnitWordCount.The actual translatable text count can be obtained by subtracting all of the above categories from the TotalWordCount, with the exception of LeveragedMatchedWordCount, FuzzyMatchedWordCount and RepetitionMatchedWordCount. These last three categories can be used to qualify the translation count itself.
The following auto text categories are applicable to text from unqualified text units with the exception of fuzzy matched text units.
The following word counts will be provided by symbolic name:
SimpleNumericAutoTextWordCount10.ComplexNumericAutoTextWordCount10,000.00.MeasurementAutoTextWordCount10.50 mm. Measurement values take precedent over the above numeric categories. No double counting of these categories is allowed.AlphaNumericAutoTextWordCountAEG321.DateAutoTextWordCount25 June 1992.TMAutoTextWordCount"Weapons of Mass DestructionTM".The following are examples of how to use the user defined extension mechanism for auto text word count categories defined in the Section 3.1::
x-OtherAutoTextWordCount'x-' and ends with the text OtherAutoTextWordCount. This is an extension mechanism.The following character counts will be provided by symbolic name:
TotalCharacterCountPunctuationCharacterCountWhiteSpaceCharacterCountProtectedCharacterCount'protected', or otherwise not translatable (XLIFF text enclosed in <mrk mtype="protected"> elements).ExactMatchedCharacterCountLeveragedMatchedCharacterCountRepetitionMatchedCharacterCountFuzzyMatchedCharacterCountAlphanumericOnlyTextUnitCharacterCountNumericOnlyTextUnitCharacterCountMeasurementOnlyTextUnitCharacterCountThe following is an example of the use of the extension mechanism for user defined character counts defined in the Section 3.1:
x-OtherNonTranslatableTextUnitCharacterCount'x-' and ends with the text NonTranslatableTextUnitCharacterCount.x-TranslatableTextUnitCharacterCount'x-' and ends with the text TranslatableTextUnitCharacterCount. The actual translatable text count can be obtained by subtracting all of the above categories from the TotalCharacterCount, with the exception of LeveragedMatchedCharacterCount, FuzzyMatchedCharacterCount and RepetitionMatchedCharacterCount. These last three categories can be used to qualify the translation count itself.
The following auto text categories are applicable to text from unqualified (see Section 2.14.2 Unqualified Text Units) text units.
The following character counts will be provided by symbolic name:
SimpleNumericAutoTextCharacterCount10.ComplexNumericAutoTextCharacterCount10,000.00.MeasurementAutoTextCharacterCount10.50 mm. Measurement values take precedent over the above numeric categories. No double counting of these categories is allowed.AlphaNumericAutoTextCharacterCountAEG321.DateAutoTextCharacterCount25 June 1992.TMAutoTextCharacterCount"Weapons of Mass DestructionTM".The following is an example of the use of the extension mechanism for user defined auto text character counts defined in the Section 3.1:
x-OtherAutoTextCharacterCount'x-' and ends with the text AutoTextCharacterCount.The following counts will be maintained for non-linking inline elements by symbolic name:
TranslatableInlineCountPlease refer to Section 2.11. Inline Element Counts for a detailed explanation and examples for this category.
The following count will be maintained for inline elements by symbolic name:
TranslatableLinkingInlineCountPlease refer to Section 2.12. Linking Inline Elements for a detailed explanation and examples for this category.
GMX-V is predicated on the text unit level of granularity (see 2.1. Text Unit). The text unit count encompasses the total number of identifiable text units in the document being counted . Any segmentation should be detailed in an SRX (Segmentation Rules eXchange format) compliant document.
TextUnitCountThe following other counts can be provided by symbolic name:
FileCountPageCountScreenCountx-OtherCountCategories'x-'. This is an extension mechanism.The following project specific counts can be provided by symbolic name:
ProjectRepetionMatchedWordCount ProjectFuzzyMatchedWordCount ProjectRepetionMatchedCharacterCount ProjectFuzzyMatchedCharacterCount The ProjectRepetionMatched counts relate to identical text units that are repeated within a project. The First occurrence of the text unit is not counted. Only the second and subsequent occurrences are counted within the ProjectRepetionMatched counts. In a similar fashion the ProjectFuzzyMatched counts also relate to fuzzy matched counts that are repeated. The first base occurrence of the fuzzy text unit is not counted, only subsequent matches.
A minimum conformance level will encompass the provision of the following categories of GMX-V for translation related tasks:
Over and above the minimum level of conformance for translation related tasks it is up to the tool supplier to provide the level of detail that is required from their product.
It is recommended that the full levels of detail are provided for both word and character counts, although it is acknowledged that this may depend on the capabilities of individual tools. For instance a given tool may not support auto text (see Section 2.17. Auto Text) and so would not be able to support auto text count categories (see 3.5. Automatically Treatable Text Character Count Categories).
For non translation related Global Information Management tasks there is no minimum level of conformance apart from the need to provide at least one count metric.
Any measurement standard must have a reference implementation as well as an authoritative body that tests and validates the measuring instruments. In the USA, this is provided by the National Institute of Standards and Technology. In order to be successful, GMX-V must provide for a certification authority that will (1) maintain reference documents with known metrics and (2) provide an online facility to test given XLIFF documents. In this way, both customers and suppliers can be safe in the knowledge that GMX-V provides an unambiguous and reliable way of quantifying a Global Information Management task.
The validity of both the embedded namespace version and the stand alone version of GMX-V metrics can be validated by means of any standard XML parser. The stand alone version cannot be validated regarding word and character counts as the text to be counted is not present.
The GMX-V document structure is designed to exist as a namespace so that it can be embedded into any document.
GMX-V can also be used in a stand alone XML document that expresses the metrics for an individual file or resource, or for a whole project. When used in a stand alone document it is recommended that the '.gmx' extension is used.
GMX-V comprises the following elements:
metricsstagecount-groupcount-group elements for verifiable and non-verifiable counts.countcount elements.projectproject element allows the grouping of
of counts for individual resource components into one
project count.resourceresource element is used to hold the
counts for individual project resources when accumulating metrics for an overall project count.GMX-V can be used to define the metrics for an individual file or resource, or for an accumulation of resources at the project level.
The following is an example of a GMX-V instance for a single file/resource:
<metrics:metrics version="1.0" source-language="en-GB" tool-name="XYZ Tool" tool-version="1.23">
<metrics:stage phase="initial" date="20041218T13:06:52Z">
<metrics:notes from="auser@company.com">
Initial count based on source document.
</metrics:notes>
<metrics:count-group name="non-verifiable">
<metrics:count type="OC-TestingFiles" value="99"/>
<metrics:count type="OC-DTPFiles" value="99"/>
<metrics:count type="ScreenCount" value="99"/>
</metrics:count-group>
<metrics:count-group name="verifiable">
<metrics:count type="TotalWordCount" value="99"/>
<metrics:count type="TotalCharacterCount" value="99"/>
<metrics:count type="TranslatableLinkingInlineCount" value="99"/>
</metrics:count-group>
</metrics:stage>
</metrics:metrics>
The following is an example of a GMX-V instance for a project within a stand alone document, where the use of the metrics namespace in not required:
<metrics version="1.0" source-language="en-GB" tool-name="XYZ Tool" tool-version="1.23">
<project identifier="Project ABC">
<resource identifier="file-1.abc">
<stage phase="initial" date="20041218T13:06:52Z">
<notes from="auser@company.com">
Initial count based on source document for the first resource.
</notes>
<count-group name="non-verifiable">
<count type="OC-TestingFiles" value="99"/>
<count type="OC-DTPFiles" value="99"/>
<count type="ScreenCount" value="99"/>
</count-group>
<count-group name="verifiable">
<count type="TotalWordCount" value="99"/>
<count type="TotalWordCount" value="99"/>
<count type="ProjectRepetionMatchedWordCount" value="99"/>
<count type="ProjectFuzzyMatchedWordCount" value="99"/>
<count type="ProjectRepetionMatchedCharacterCount" value="99"/>
<count type="ProjectFuzzyMatchedCharacterCount" value="99"/>
</count-group>
</stage>
</resource>
<resource identifier="file-2.abc">
<stage phase="initial" date="20041218T13:06:52Z">
<notes from="auser@company.com">
Initial count based on source document for the first second resource.
</notes>
<count-group name="non-verifiable">
<count type="OC-TestingFiles" value="99"/>
<count type="OC-DTPFiles" value="99"/>
<count type="ScreenCount" value="99"/>
</count-group>
<count-group name="verifiable">
<count type="TotalWordCount" value="99"/>
<count type="TotalWordCount" value="99"/>
<count type="ProjectRepetionMatchedWordCount" value="99"/>
<count type="ProjectFuzzyMatchedWordCount" value="99"/>
<count type="ProjectRepetionMatchedCharacterCount" value="99"/>
<count type="ProjectFuzzyMatchedCharacterCount" value="99"/>
</count-group>
</stage>
</resource>
</project>
</metrics>
Please note that there are a variety of count types of which only some basic examples are present in the examples above. A full list of count types is provided in the specification of the <count> element type attribute.
There are four type attributes that are specific to project specific counts:
ProjectRepetionMatchedWordCountProjectFuzzyMatchedWordCountProjectRepetionMatchedCharacterCountProjectFuzzyMatchedCharacterCountThe ProjectRepetionMatched counts relate to identical text units that are repeated within a project. The First occurrence of the text unit is not counted. Only the second and subsequent occurrences are counted within the ProjectRepetionMatched counts. In a similar fashion the ProjectFuzzyMatched counts also relate to fuzzy matched counts that are repeated. The first base occurrence of the fuzzy text unit is not counted, only subsequent matches.
The <metrics> element is the top level of the hierarchy. It signals the start of the metrics namespace DOM tree. Its direct children are one or more <stage> elements, or one or more <project>
elements when exchanging project based metrics. It is possible to maintain metrics for one or more stages, or one or more projects.
The <project>
element is used to accumulate the individual <resource> elements for a specific count when exchanging metrics for a whole project. This element is only used for project based counts.
The <resource> element holds the individual <stage> elements with the specific counts when exchanging metrics for a single file in a project. This element is only used for project based counts.
The <stage> element is used hold the <count-group> elements for a specific count stage, as well as one or more optional <notes> elements.
The <notes> element is used hold optional comments about the metrics stage.
The <count-group> element is used to contain verifiable or non-verifiable <count> elements.
The individual <count> elements hold the values of the count and identify the type of count.
The GMX-V document structure can exist either as a namespace within another document such as an XLIFF document, or within a standalone file. If GMX-V data is embedded within another document then a namsepace declaration will be required. The mandated namespace to be used is 'metrics'.
The GMX-V namespace declaration has the following form:
xmlns:metrics="urn:lisa-metrics-tags"
If the GMX-V document exist within its own file, then by convention the file should have a '.gmx' extension. No namespace declaration is required in this instance.
| Elements | <metrics>,
<project>,
<resource>,
<stage>,
<notes>,
<count-group>,
<count> |
The topmost GMX-V element has the following format:
<metrics>
The <metrics> element encloses all the other GMX-V elements of the document.
Required attributes:
version - the fixed GMX-V current version identifier, currently "1.0".
source-language - the language in which the document is authored.
tool-name - The name of the tool that generated the metrics.
tool-version - The version identifier of the tool that generated the metrics.
Optional attributes:
target-language - the target language for the document. Only relevant if any translation memory matching has been done for a particular target language.
reference - a reference to an external file or identifier if the metrics are not embedded in the file that is being counted.
Contents:
One or more <stage> elements, or one or more <project> elements.
The optional project element has the following format:
<project>
The <project> element can be used to group individual <resource> counts to provide a metric count for a complete project.
Required attributes:
NONE
Optional attributes:
identifier - the project identifier.
One or more <resource> elements.
The resource element has the following format:
<resource>
The <resource> element contains the <stage> count elements for each resource.
Required attributes:
identifier - the resource identifier, e.g. the file name of the resource.
Optional attributes:
NONE
One or more <stage> elements.
The Stage element has the following format:
<stage>
Required attributes:
phase - The identifier for the stage.
date - the date that the stage count was created.
source-language - the source language for the stage.
Optional attributes:
target-language - the target language for the stage.
Contents:
One or more <count-group> elements.
The Notes element has the following format:
<notes>
Required attributes:
NONE
Optional attributes:
from - The email address or other identifier of the creator.
date - the date that the notes element was created.
Contents:
Comments text, no elements.
The Count Group element has the following format:
<count-group>
Required attributes:
name - The count group name. This will have two possible values: verifiable or non-verifiable.
Optional attributes:
state - The count group state. This can be used to create count groups for different states during translation.
Contents:
One or more <count> elements.
The Count element has the following format:
<count>
Required attributes:
type - The count type.
value - The quantity value.
Optional attributes:
category - The fuzzy match count category, e.g. "93-95".
Contents:
EMPTY
This section lists the attributes used in the metrics elements. An attribute is never specified more than once for each element. Along with some of the attributes are the "Recommended Attribute Values". Values for these attributes are case sensitive. These lists are purely informative; the goal is to specify a preferred syntax so tools can have some level of compatibility.
attributes |
category,
date,
from,
identifier,
name,
phase,
reference,
source-language,
state,
target-language,
tool-name,
tool-version,
type,
value,
version
|
the category of fuzzy match. This is the percentage category within which the match falls, e.g. "99-95".
Value description:
The fuzzy match category value.
Default value:
Undefined
Used in:
The date attribute indicates when a given element was created or modified.
Value description:
Date in [ISO 8601] Format. The recommended pattern to use is: YYYYMMDDThhmmssZ
Where: YYYY is the year (4 digits), MM is the month (2 digits), DD is the day (2 digits), hh is the hours (2 digits), mm is the minutes (2 digits), ss is the second (2 digits), and Z indicates the time is UTC time. For example:
date="20020125T210600Z" is January 25, 2002 at 9:06pm GMT is January 25, 2002 at 2:06pm US Mountain Time is January 26, 2002 at 6:06am Japan time
Default value:
Undefined.
Used in:
the project or resource identifier.
Value description:
The identifier for the project or resource.
Default value:
Undefined
Used in:
The email address or other identifier of the creator of a given notes element.
Value description:
The identifier of the creator of this notes element.
Default value:
Undefined
Used in:
The name of the count-group.
Value description:
Must have the value verifiable or non-verifiable.
Default value:
Undefined
Used in:
The phase name of the stage.
Value description:
Can have the value initial, final or user defined.
Default value:
Undefined
Used in:
An identifier to the external file or identifier if the metrics relate to an external file.
Value description:
The file name or other identifier relating to an external file to which the metrics relate.
Default value:
Undefined.
Used in:
The language for the main <metrics> element.
Value description:
A language code as described in the [RFC4646]. For more information see the section on xml:lang in the XML specification, and the erratum E11 (which replaces RFC 1766 by RFC4646).
Default value:
Undefined.
Used in:
State - The optional count-group state qualifier. Separate
count-group elements can be maintained for the different states of the
target elements that correspond to the counted source
element content in an XLIFF file.
Value description:
The pre-defined values are based on the state
attribute values from the XLIFF specification document.
| Value | Description |
|---|---|
final |
The count-group for XLIFF trans-units with target elements with a status attribute of 'final'. |
needs-adaptation |
The count-group for XLIFF trans-units with target elements with a status attribute of 'needs-adaptation'. |
needs-l10n |
The count-group for XLIFF trans-units with target elements with a status attribute of 'needs-l10n'. |
needs-review-adaptation |
The count-group for XLIFF trans-units with target elements with a status attribute of 'needs-review-adaptation'. |
needs-review-l10n |
The count-group for XLIFF trans-units with target elements with a status attribute of 'needs-review-l10n'. |
needs-review-translation |
The count-group for XLIFF trans-units with target elements with a status attribute of 'needs-review-translation'. |
needs-translation |
The count-group for XLIFF trans-units with target elements with a status attribute of 'needs-translation'. |
new |
The count-group for XLIFF trans-units with target elements with a status attribute of 'new'. |
signed-off |
The count-group for XLIFF trans-units with target elements with a status attribute of 'signed-off'. |
translated |
The count-group for XLIFF trans-units with target elements with a status attribute of 'translated'. |
In addition, XLIFF user-defined values can be used with this attribute. A user-defined value must start with an "x-" prefix.
Default value:
Undefined.
Used in:
The target language for the main <metrics> element.
Value description:
A language code as described in the [RFC4646]. For more information see the section on xml:lang in the XML specification, and the erratum E11 (which replaces RFC 1766 by RFC 4646).
Default value:
Undefined.
Used in:
The identifier of the tool used to create the metrics.
Value description:
the name of the tool used to perform the metrics count.
Default value:
Undefined
Used in:
The version identifier of the tool used to perform the metrics count.
Value description:
the version identifier of the GMX-V count tool.
Default value:
Undefined
Used in:
The count type.
Value description:
Can have any of the following values, or a user defined type beginning with x- as defined in Section 3.1:
ScreenCountFileCountPageCountTextUnitCountTotalWordCountWhiteSpaceCharacterCountPunctuationCharacterCountAlphanumericOnlyTextUnitWordCountMeasurementOnlyTextUnitWordCountNumericOnlyTextUnitWordCountExactMatchedWordCountLeveragedMatchedWordCountRepetitionMatchedWordCountProtectedWordCountProjectRepetionMatchedWordCountFuzzyMatchedWordCountProjectFuzzyMatchedWordCountTotalCharacterCountAlphanumericOnlyTextUnitCharacterCountMeasurementOnlyTextUnitCharacterCountNumericOnlyTextUnitCharacterCountExactMatchedCharacterCountLeveragedMatchedCharacterCountRepetitionMatchedCharacterCount ProjectRepetionMatchedCharacterCountFuzzyMatchedCharacterCountProjectFuzzyMatchedCharacterCountProtectedCharacterCountSimpleNumericAutoTextWordCountComplexNumericAutoTextWordCountMeasurementAutoTextWordCountAlphaNumericAutoTextWordCountDateAutoTextWordCountTMAutoTextWordCountSimpleNumericAutoTextCharacterCountComplexNumericAutoTextCharacterCount MeasurementAutoTextCharacterCountAlphaNumericAutoTextCharacterCountDateAutoTextCharacterCountTMAutoTextCharacterCountTranslatableInlineCountTranslatableLinkingInlineCountDefault value:
Undefined.
Used in:
The numeric value of the count record.
Value description:
The value of this count.
Default value:
0
Used in:
The current GMX-V version number.
Value description:
The version number of this metrics document:
Fixed value:
1.0
Used in:
The following figure shows the possible structure as a tree. Each element is followed by notation indicating its possible occurrence according to the corresponding legend.
(legend: 1 = one
+ = one or more
? = zero or one
* = zero, one or more)
<metrics>1
|
+--- <project>+
| |
| +--- <resource>+
| |
| +--- <stage>+
| |
| +--- <notes>?
| |
| +--- <count-group>+
| |
| +--- <count>+
|
+--- <stage>+
|
+--- <notes>?
|
+--- <count-group>+
|
+--- <count>+