Symbian developer community

 
Home > Wiki> Text & Localisation Services > Backlog > Backlog Details
wiki

Text & Localisation Services/Backlog/Backlog Details

From Symbian Developer Community

Jump to: navigation, search

Contents


Installable Fonts

Background

For many years, a key feature missing in Symbian OS was the ability to install fonts at run-time, just like on a desktop OS. This was in no way due to any drawbacks in the OS or the OS architecture itself. Symbian OS is perfectly suited to allow fonts to be installed at run-time, without the need to restart the device. This could perhaps be a key use experience and personalisation feature in the future versions of Symbian.

3rd party application developers have for while now figured out how to force-install fonts into Symbian OS, either for application use, or to bypass the native System Font. Font installation for specific application use is more common, as fonts can come packaged with application SIS files and be installed to the same location as the applicaition binaries. However, fonts cannot yet be installed into the system fonts folder, which contains the System UI font and all the fonts used by the default S60 applications. Due to this, the System UI font cannot be easily by-passed, although 3rd party developers have found ways to do so.

Platform enablers

As I mentioned, not much needs to be done in the platform to enable font installation.

The main area for change is the part that loads and catelogues fonts that are present in the \resource\fonts\ directory. Currently, the Font and Bitmap Server (Fbserv) loads all the fonts in the \resource\fonts\ folder at startup (boot-up). The Fbserv is one of the first servers to start during device boot. It does this by searching through the folder looking for valid font files, and using the Rasterizer instantiates font objects representing the physical font file, it gives owenrships of the fonts to the Font Store (FntStore). This way Fntstore builds a list of all fonts in the system.

So it seems obvious that for font installation to work on the fly, Fbserv needs to be able to 'load' fonts as they are installed into the \resource\fonts\ folder, and give ownership of those fonts to Fntstore. For this, Fbserv needs to watch the fonts folder for additions.

The process that handles the System Font higher up in the OS will need some sort of a trigger if a new font is installed for System Font replacement, to ask the user if he/she wants to replace the UI font. This can perhaps be done in the form of a notification from Fbserv when it installs a new font.

Once you enable font installation, you would probably want to enable font removal. This is a bit trickier than font installation, in that if a user decides to remove a font that is being used as the UI font (System Font), then some mechanism has to be put in place which gracefully replaces the UI font with a fallback font and then uninstalls the font. However, this cannot be done by the Fbserv. The Fbserv can only remove a font if it is not being used by any process. It keeps track of this using its in-built access count. Only when the access count is 0, will it be able to remove a font. The Font Provider in the higher layer of the OS that maintains the System Font would need to do the UI font replacement before it initialises a font removal.

Font installation and selection UI, and its challenges

The font selection UI is perhaps the main place (or the only place) where a user can download a new font and then replace the UI font (or System Font in engineering terms) with the downloaded font. The font selection UI can also be used just to switch the UI font with an existing one (i.e. without downloading a new one). The UI would probably look something like this:

File:Font_Selector_UI_Example.png


The 'Download new font' option would probably link to a server where fonts can be downloaded seamlessly (some sort of an application store ). However, it would be possible to install a 3rd party font via a SIS package.

Here is where it gets interesting. Security could be an issue when installing a 3rd party UI font. When the font is downloaded and installed, some sanity checks etc. will be done by Fbserv in order to check if it is a compatible font etc. However, the font could be a rogue font that, for e.g. could have empty boxes as glyphs for all characters. If the user switches to using this font as the UI font, it could render the device useless (or at the least cause annoyance). For this problem, some sort of checks need to be provided by the font selection application. Perhaps it could give the user a 10 second preview of the UI with the new font, during which the user has to press accept, and after which the device will switch back to using the original UI font. Of course, there are other ways to ensure that user isn't left dry by downloading a rogue or corrupt font. Perhaps more comprehensive checks could be done by the Fbserv before installing a font.

Font metrics for System Font

A mobile device interface has very limited real estate. For this reason, font sizes are carefully chosen to allow optimal usage of this real estate and provide excellent user experience when it comes to text on the UI (such as in menus, on the homescreen etc.). This is done separately for all languages supported by the device, as translations of menus etc. have different lengths in different langauges. Hence to enable 3rd party fonts to look good on Symbian devices, the font metrics for the UIs will need to be published. 3rd party font vendors wishing to create fonts for Symbian devices can use those font metrics to customise fonts for Symbian OS.

New Symbian Locale Model

Locales in General

Locales are nothing but a set of parameters of settings that define users' language and country preferences, and also certain variant preferences that either the user or the operator wants on the UI.

These seem like trivial user settings which define what UI language is in use, or which country the user is in, so that application such as Clock can set the country specific settings etc. However, in todays world of device personalisation, locales can play a powerful role in allowing the user to control how applications and even services behave when used on the user's device.

Take for example a simple setting that defines which language the UI is in. An application like Maps, when downloading maps onto the device, can read this setting, and regardless of the location of the user, download maps and services in that particular language, rather than the default language spoken in the country where the user is located at the time. Social networking apps can also use the locale language setting to display information in the users preferred language.

The country setting in locales can, for example, be used by services apps that display financial information and can use the users currency setting to change the currency in which the financial information is displayed. The country setting can also define postal address formats that can be used by services apps that need to show localised address formats the user understands.

Current Locale Model

Locales in Symbian are data DLLs named elocl.XX, XX being the Symbian OS language ID as defined in the TLanguage enumeration in e32lang.h (exported header file). The settings in Symbian Locales can be categorised into 3 basic groups - Language, Country and Collation.

Locales are loaded at startup and the settings read and publshed for applications and services to use. Subsequent user-initiated changes to locale settings re-publishes the settings.

The Symbian Locale Model as two interfaces. A client interface and a supplier interface.

The client interface is a set of APIs that applications and services can use to read locale settings and load new locales to change the current device locale. There are also APIs that allow applications to change individual locale settings, in affect allowing the user to create custom locales.

The supplier interface is for device vendors to create locales, and compile locale DLLs. Since a locale is a group of settings, each device vendor would probably have different values for those settings for any particular locale (for e.g. English_GB etc.). Some settings would also vary according to operator preferences.

The supplier interface is a single interface that covers all the locale settings, i.e. language, country and collation. Hence device vendors who want to create locales such as English_GB, English_US and English_CA, need to implement the entire interface 3 times, and each locale DLL created will have a single unique identifier and suffix taken from the TLanguage enumeration (elocl.01, elocl.02 ... elocl.98 etc.)

File:Symbian_Locale_Model.PNG

The problem with the current Locale Model

The current Symbian Locale Model is old, inconsistent and inflexible. The key issues with respect to these shortcomings detailed below.

Wrong categorisation of locale settings

As mentioned above, the supplier side locale interface is a single interface with all setting pertaining to language, country, collation etc. mixed. The client interface, with classes like TExtendedLocale to some extent attempts to group and categorise these settings into the above mentioned categories, so that settings applications can be consistent in changing locale settings when the user changes the phone language or the country. However, it does so very loosely and in some cases wrongly as well.

The current groups of settings, or 'aspects' as they are called in Symbian, are vague, i.e. as a category, they don't provide the user with any useful personalisation options. One such aspect is the 'LocalelocaleAspect' aspect, which is a collection of orphan settings from the older Symbian locale model, and doesn't really make any sense.

Other settings are simply mis-categorised into the wrong aspect, and hence from a user perspective, changing that group of settings would yiled the wrong results. Some examples of this are language and country settings being mixed up.

Inflexible Supplier Interface

Today, locales in Symbian are monolithic data DLLs, where groups, or aspects of settings are compiled into a single binary. This means that if 2 locales on a device share the same aspect, the settings would have to be replicated in each locale to complete that locale. This not only wastes ROM space (and disk space when Language Installation is enabled) because of settings replication, but also makes managing locales much harder.

Here's a scenario that covers both the issues:
Arabic is spoken in many countries in the Middle East. This would mean that the group of Arabic language settings would be common across many countries, even if the Country (or Region) settings are different. However, due to the monolithic nature of current Symbian locales, these language settings must be replicated across all the locales that are made up of Arabic language settings, and each locale must be given a separate TLanguage ID. On a device, each of these locales would unnecessarily occupy space, as the language settings cannot be reused.

In terms of managing locales, this would not only mean managing extra locales with unique locale IDs, but it would also mean that if 1 setting in the Arabic language group of settings needs to be changed, say for a network operator operating across the Middle East, then that setting would need to manually be changed across all the locales with Arabic language settings. This is a logistical nightmare!

For e.g. ELangSindhi_Arabic and ELangSindhi_Devanagari, OR ELangEnglish_Apac, ELangEnglish_Taiwan, ELangEnglish_HongKong, ELangEnglish_Prc, ELangEnglish_Japan and ELangEnglish_Thailand

Symbian locales are 'language' centric

As you may have gathered from above, locales in Symbian are one-dimensional, i.e. even though they contain 'country' or 'region' setttings, and collation settings, they are named according to the language settings they contain, and are given IDs that were created to represent 'language' in Symbian OS (TLanguage). In other words, language, country and collation are not separated. There is no enumeration called TCountry or TRegion, that represents the region settings of locales.

Symbian locales are DLLs

As mentioned many times above, locales in Symbian are compiled, binaries, or data DLLs. This was done for 2 reasons:
1. Locales are critical for phone startup, and DLLs are guaranteed to be loaded. Non-binary locales are not guranteed to be laoded
2. Symbian locales contain collation keys, which can get quite large, for languages such as Chinese etc. Loading compiled keys is faster.

However, this causes 2 problems:
1. The DLL's have to be built at some time during phone creation, and this is mostly done along with the rest of the binaries very early on in the phone creation process. However, since locales are essentially configuration settings, building them early on is a wasted effort, as configuration settings usually change during the final processes of phone creation. This makes creating localised devices highly inefficient.
2. Standard locales cannot be used in Symbian OS: Symbian is an open source OS, and one of our missions is to use open source standards that are becoming industry standards as well. Locales standards like POSIX and CLDR by Unicode provide ready-to-use locales that in theory could simply be put on to a device and reduce the cost for device manufacturers. However, because Symbian locales are binary and implement a proprietary interface, these standards cannot be used.

The solution - The New Symbian Locale Model

A new locale model for Symbian was proposed, and in the process of being implemented, to rid device creators off some of the problems mentioned above, to to also provide the end user with greater flexibility in cnfiguring his/her device.

The following changes will be brought to the Symbian Locale Model for Symbian^4

Re-categorising locale settings in 'aspects'

The first part of re-modelling the locale model is to put the locale settings in the right aspects, or categories, or groups, removing some aspects that don't provide any benefits to the user for personalisation. The end goal is to end up with 3 aspects:

Language: All settings that will get changed on the device when the user switches the UI language
Country: All settings that will get changed on the device when the user switches the country he/she is in
Collation: The locale character set used for collation will be changed when the user changes this setting. This can be done either for the entire device, or for a particular application. It is up to the Settings application whether to expose this setting directly to the user

Splitting the Locale Creation Interface according to the 3 aspects

The biggest change by far in the locale model will be to split the locale creation (supplier) interface for creating locale DLLs. As mentioned above, locales are currently monolithic DLLs implmenting a single interface, causing the infelixibilities mentioned above.

The interface will be split into the 3 aspects - Language, Country and Collation. Each interface will be implemented separately, which means there will be 3 DLLs representing a single locale. This is a big change, and constitutes a binary break in the locale creation interface, and has many consequences on the rest of the Symbian locale concepts and code.

The consequence of this change would be the introduction of a new identifier for the new DLLs. Up till now, the language ID (TLanguage enumeration) was used as locale IDs, thus not allowing locales to be differentiated by the country. To get around this, Nokia introduced new TLanguage IDs that allowed differentiation by country for locales that had common langauges, for e.g. ELangSindhi_Arabic and ELangSindhi_Devanagari, OR ELangEnglish_Apac, ELangEnglish_Taiwan, ELangEnglish_HongKong, ELangEnglish_Prc, ELangEnglish_Japan and ELangEnglish_Thailand.

By splitting the interface and introducing a Country and Collation interface, the new DLLs will need a new type of unique ID. The language DLLs will still use the TLanguage IDs (TLanguage will become a pure 'language' enumertion). The collation DLL, since it is based on language, will also use TLanguage enumeration as its identifier.

New TCountry enumeration

For the Country DLL, a new enumeration TCountry will be introduced, thereby for the first time introducing the region setting in Symbian OS. Thid enumeration will be used as the Country DLL suffix as well. Since there is no legacy issues or backward compatibility issues with the country settings, we can for the first time introduce standardised naming conventions in Symbian OS. Hence the TCountry enumeration will use the ISO 3166 country codes.

It should be noted here that we are now introducing more DLLs into the system for locales. A single complete locale will no comprise of 3 DLLs, and a device would have to have at least 1 DLL each of Language, Country and Collation. However, this doesn't mean that there would be more DLLs than before to manage. There would be in fact fewer DLLs to manage, since most of the DLLs would be common across many locales. For e.g., there would only be 1 Arabic language DLL (except when there is an actualy variation of the Arabic language) for multiple Country DLLs.

From a device point of view, each part of the locale, i.e. a Language, Country or a Collation DLL, will be loaded separately during startup, and then again when the user changes the system preferences. The settings from these DLLs can be read individually, and published to the system individually, therefore giving maximum flexibility in the system and to the user.

New APIs for loading locales

New APIs will be needed in order to help load the new locale DLLs, as the older APIs to load a single monolithic DLL wont be useful. The new APIs are as follows:

API Usage
TInt TExtendedLocale::LoadLocale(Language ID, Country ID, Collation ID); This API will load a complete locale, i.e. 3 DLLs Language, Country and Collation. If any one DLL is missing, it will not load any of the others, as the locale is incomplete
TInt TExtendedLocale::LoadLocaleAspect(Locale aspect <language, region, collation>); This API will load an aspect, i.e. 1 DLL either Language, Country or Collation
TCountry User::Country(); User level API that returns the current country setting

Backwards compatibility

This is a very tricky part of the solution. In Symbian OS, we always try to ensure that all our changed are backwards compatible, and that no application developer, device creator, or device user should adversly suffer as a result of a new feature or a feature improvement.

The Symbian locale framework has 2 main users - application developer and device creator. Hence backwards compatibility needs to be kept for both these users to ensure existing applications and existing locales continue to work in newer versions of Symbian.

Backwards compatibility will be kept in stages, with the first stage strictly allowing the old mapping between old APIs and old locales to work only, i.e. old APIs work only with old locale DLLs and new APIs work only with new locale DLLs.

This is an interim stage, as when the locale creators decide to use the new interfaces to replace all the old locale DLLs with the new ones, the old APIs would be broken. At this point, the next stage of backwards compatibility would be implemented. The old APIs would be re-routed to the new ones, and a simple adaptation would be implemented that converts the parameters of the old APIs (single locale DLL name - 3 aspect DLLs) into parameters of the new APIs.

Gurumukhi and Malayalam Writing System Rendering Support

Background

Gurumukhi is the writing system, or script used in the Punjabi language, spoken in the northern Indian state of Punjab and some parts of Pakistan. The writing system for Punjabi used in Pakistan however is a form of Arabic called Nastaliq.

Punjabi speakers can also be found in small numbers is all parts of India. There is a significant population of Indian Punjabi speakers in the United Kingdom and Canada as well, and hence the market for Gurumukhi enabled phones is quite significant if you take all these regions into consideration.

Malayalam is the writing system, or script used in the Malayalam language, spoken predominantly in the southern Indian state of Kerala, and in some union territories of Lakshwadeep and Mahe. Besides these places in India, Malayalam is also spoken in places such as the Persian Gulf, United States, Singapore, Australia, and Europe, significantly adding to the market for Malayalam enabled phones.

For more information on these scripts, please visit:

Rendering Support

To enable rendering support in Symbian OS for he above scripts, and other Indic scripts, very few changes need to be made to the rendering code in GDI (soon to be moved to a new component) and the open source ICU Layout Engine shaper that is shipped with Symbian OS.

The GDI rendering code needs to recognise the Unicode ranges of the scripts and divert the rendering function call to static functions declared in script specific classes. The functions are named [script class name]::ProcessFunction(). All this happens inside the powerful generic text rendering static API called CFont::GetCharacterPosition2().

Other changes to GDI include adding the script specific static functions to do the rendering, and adding state machines for every Indic script to implement the rendering rules for the scripts as defined in the ISCII 1991 standard. The state machine splits the text into syllables, that allows for correct cursor movement between syllables, and also syllable caching in the FontStore Shaper Cache for faster Indic text rendering. The syllables are then collected and sent to the shaper, which in the default case is the ICU Layout Engine.

The changes needed in the ICU Layout Engine depend on how accurate the default open source implementation of the script rendering is. The version of ICU Layout Engine currently used in Symbian is 3.4, which is very old, and still not mature enough to handle a lot of the complex ISCII rules. For this reason, the ICU Layout Engine needs to be thoroughly tested for these rules, and changes need to be made to it accordingly. Changes include additional sates to the internal ICU Layout Engine state machine used for syllable splitting, and also script specific glyph substitution rules.

Language-Based Glyph Variant Substitution/Selection

The problem

Text rendering in Symbian OS has no concept of 'language' today. At the platform level, all text is rendered as Unicode, and Unicode only identifies and separates text according to 'writing systems' or scripts. Latin is a writing system/script, and so is Arabic etc. Where is 'English' is a language that is based on the Latin script, and likewise 'Urdu' is a language based on the Arabic script.

Generally speaking, the language in which a script is used does not dictate the look and feel of the characters, or glyphs of the that script. An 'e' written in French looks exactly the same as an 'e' written in English. Of course, this should not be confused with the various 'styles' and 'typefaces' in which an 'e' can be written (italic, bold, Times New, Courier New etc.). Those are not language variations, but rather style or look & feel variations.

However, there are a few scripts defined in Unicode, few but important scripts, whose glyphs vary based on the language that is using those scripts. Some simple examples of this are Urdu and Farisi, whose glyphs for Arabic digits differ from each other, or Hindi and Marathi, whose glyphs for the Devanagari characters 'Ra' and 'La' differ from each other. However, these examples aren't significant enough to have much of an impact on the way these scripts are rendered.

The main example of this is the Han characters in CJK (Chinese, Japanese and Korean) scripts. Han is a common character set amongst the CJK languages, and each country using these Han characters had separately defined encodings for those characters for their countries. Unicode 2.0 sought to unify these Han ideographs, and even though those ideoraphs looked different when used in different languages. Hence to be perfectly clear, the differences were not in the way fo e.g. , traditional Chinese characters are different from simplified Chinese characters. Han unification took place where the same character was written differently in Japanese or Chinese, because different typographical or glyph design rules existed in these countries. These unified Han ideographs are collectively known as Unihan.

File:Han_glyph_variants.PNG

Han unification however brought with it problems. Since the different variants of the glyphs now all shared a common codepoint, software systems rendering these glyphs had to find a way to distinguish between them, so that the correct glyph for the correct language could be rendered. This problem is often known as Unihan ambiguity. The result of this problem is that Chinese, Japanese and Korean text using Unihan characters cannot be rendered together, on web pages, text messages, emails etc. etc., unless the rendering software can distinguish between the language in which the client is attempting to render the text.

The solution

The solution to this is pretty straighforward, but is made difficult due to the primitive and inflexible architecture of the Symbian text & font support. The solution would have two parts:

Part 1: 'Language' as a platform-level variation point:
As mentioned before, Symbian OS does not have a concept of 'language' or 'language system' on the platform. So the first part of the solution is to somehow plumb in a UI customisable setting through to the low-level text rendering code that relays the 'language' in which the text needs to be rendered. This is by no means an easy task, as text control in Symbian OS goes through various interfaces and classes, most of which are inflexible to accomodate a new 'setting' for language system. In 2006, I had made a proposal for how this could be implemented. The proposal is here

Of course, since then I have learned a lot more about the text rendering architecture in Symbian OS, and also about text rendering in general, so I wouldn't be surprised if there are better ways to implement a language system setting in Symbian OS . In fact, I myself have had a few more ideas which I have mentioned below:

  • Language 'tags' embedded in the text itself: This is a bit ambitious, but with some clever tweaking of the FORM component (that formats text), we can embed Unicode Language Tag inside the Unicode text (using Unicodespecified LANGUAGE TAG characters), which will be ignored by the text formatting engine FORM, as specified by the Uniocde rules on processing LANGUAGE TAGS, but get consumed and interpreted low-level in the rendering code that already recognises the writing system/script by looking at the Unicode values. For key public text rendering API such as DrawText?, the client will need to prepare the text before hand with embedded language tags, so that the entire mixed language/script text descriptor can be rendered within a single call to DrawText? (which is what happens now).

    The concept of embedding extra information inside the text descriptor is already implemented for other use cases, and hence the conept is already proven. In 2008, new DrawText? APIs were introduced that were able to handle text which had context embedded into it. For the single use case it was implemented for, the context was used to draw that particular string along the same baseline as that of the contextual character embedded, allowing for uniform baseline rendering across different runs of text.

    File:Language_Tags.PNG
    For a detailed explanation on embedded LANGUAGE TAGS in text the rules for processing such characters, please see this Unicode publication, section 16.9 TAG Characters

From an application perspective, there are several ways in which language information can be relayed to the operating system. HTML supports language tags that can be prefixed to the text content, so that web pages that contain text in many languages can be rendered in the correct way. When the text in the web pages gets rendered, the HTML language tags can be translated by the web browser into Symbian OS specific language system setting, and can be passed on the the OS in any of the above mentioned ways (i.e. through font specification, or tags embedded in the text etc.).

Simillarly, a text editor application that allows text input in many languages (using a language FEP and an appropriate input method), can relay the language system setting information by translating the FEP setting into a Symbian OS language system setting, and again pass it on to the OS in any one of the above mentioned ways.

Here is a use case
When a user chooses and English input language, the text editor application translates this setting into a Symbian OS language system setting, and embeds an English language tag at the beginning of the text that the user types. If the user at some point decides to switch the input language, to say, Hindi, the editor then embeds a new language tag at the beginning of the text that is typed from then on.

The text editors here play an important role, because it is their responsibility to save the user setting when the document is saved, so that the rendering is done correctly when the user opens a document that was written in multiple languages. This can be done by saving the embedded Unicode language tag character with the text in the document itself. This then would pose the obvious interoperabilty question. What will happen when the document is then transfered to another device and opened there? What will happen to the embedded language tag? Will it be rendered as a garbage character? If devices implement Unicode to the book, then they would follow the Unicode rules on either ignoring language tag characters, or processing them as specified in the Unicode specifiation.

Part 2: Language based glyph-varaint selection:
Now comes the fun part. Assuming by now that the language system setting as been plumbed through the Symbian OS text rendering/font specification APIs, the next step would be to use that language system setting when mapping the incoming Unicode text to their corresponding glyphs in the specified font.

This currently happens on a 1-1 bases (except for Indic languages, which I shall discuss later), i.e. for every Unicode character encountered, the basic CMAP font table is used during Rasterization to map that Unicdoe character to its default glyph in that font.

However, fonts such as OpenType? and TrueType? ones also have tables such as the GSUB (glyph substitution) table that have a 1-many glyph mapping, that can be used to obtain a variant glyph based on certain conditions, such as 'language'. The text rendering code in Symbian OS, which currently selects the rendering process based on writing system/script (based on Uniocde codepoint identification), will need to be enhanced to take into consideration the extra parameter - language system - to decide how it wants to process the incoming text. The enahncement wil include access to the glyph tables, such as the GSUB table needed for this to work.

For Indic languages, such as Hindi, Kannada etc., the language based glyph selection is already partially implemented. By partial, I mean that the rendering code today can select glyph variants based on language, but the language system setting still needs to be plumbed through the text rendering APIs in one of the ways suggested above in point 1. Indic languages in Symbian OS are rendered using a Shaper. We currently use the ICU Layout Engine to do Indic shaping. The shaper is responsible for taking the Unicode text, doing transformaitons such as ligature formation, repositioning on charactes, contextual glyph substituion etc, and then producing a final list of glyphs corresponding to the input Unicode text. The ICU Layout Engine also supports language-based glyph substituion. The API accepts a 'language code' parameter which is used internally by ICU Layout Engine during glyph substituion (it uses the GSUB table).

However, the shaper is not used for other writing systems/scripts and languages, and hence the language based glyph selection/substituion would need to be implemented in Symbian OS code.

For more information on the concept of Language based glyph-variant selection, and for more details on the solution to the problem, please see the following links:

Arabic and Thai shaping using OpenType tables

Arabic and Thai scripts are rendered in Symbian OS using hard coded Unicode contextual substituion tables of Unicode Presentation Forms that specify context based character substituion, before that character is Rasterized and a glyph for it obtained. This limits the number of character substituions that are supported by Symbian OS, as the tables are just a subset of the exhaustive set of tables defined in the Unicode specification. This leads to inaccurate and rather substandard Arabic and Thai experience on Symbian Devices, in contrast to the excellent Arabic and Thai experince on the iPhone and Android based devices owing to the large number of character substituions supported.

A shaper, however, would do away with the static, table-based rendering of Arabic and Thai, and introduce dynamic contextual glyph substituion like it is done with Indic writing systems/scripts. The glyph substituion would ensure that the glyph selection is not limited to the ones that are currently defined in the the Symbian OS subse of the Unicode presentation forms.

What will need to be done
  • Test the ICU Layout Engine comprehensively for Arabic and Thai shaping with new unit tests (similar to Indic ones) to prove that the ICU is capable of quality * Arabic and Thai rendering
  • Make changes to the Unicode processing code for Arabic and Thai in GDI (soon to be moved to a new component) so that text is passed to the shaper
  • Run the existing GDI integration tests for Arabic and Thai to test for regressions. Fix any issue in either the ICU Layout Engine or the test code
  • Write new integration tests in GDI that test the GetCharacterPosition2?() API for aditional Arabic and Thai text sequences and characters that will automatically be supported once the rendering takes place through the shaper

Hangul (Korean) combining jamos using OpenType tables

Symbian OS today partially supports rendering of the Hangul writing system/script for Korean language, in that it is fully capable of rendering jamos (Hangul character). Jamos, like Indic characters, can combine to form other jamos, and these combined jamos have been given Unicode codepoints as well. Hence, Symbian OS, since it has a Uniocde based text renderer, can also render pre-combined jamos. However, Symbian OS cannot combine jamos like it can combine Indic and Arabic characters to form syllables. This is generally not an issue, as Korean text from outside sources (web pages, emails etc.) normally come as pre-combined jamos (as the source would have combined them before transmitting them), and get rendered in Symbian as Unicode.

However, for text editing, this is an issue, as the user wants the ability to enter single jamos to produce combined ones on the fly. Again, this issue can be circumvented by having FEPs that provide pre-combined jamos for the user to choose from, or even combine the jamos themselves in the FEP code before presenting them to the OS for rendering. But for the best rendering support, I would advise that at some point we support combining of jamos within the OS platform, rather than relying on higher layer components/assets/applications for doing that for us. Once again, like Arabic, Thai and Indic scripts, this can be done using the shaper.

What will need to be done

For shaping jamos, we would need to do pretty much what is described above for Arabic and Thai.

Retrieved from "http://developer.symbian.org/wiki/index.php/Text_%26_Localisation_Services/Backlog/Backlog_Details"

Comments

Sign in to comment…