Introduction

Magento is often described more as an application framework than a mere shopping cart. It heavily relies on coding practices derived from enterprise Java development and is thus a very powerful system. Alas, some parts of Magento were more thouroughly thought-of than others. This creates odd situations where some goals are surprisingly easy to acheive using nothing more than the administration panel's GUI interface, whereas others require heavy programming knowledge.

Magento's localization system is one such area that seems to have been constructed without a lot of planning. In particular, the variety of methods used here is indicative of the fact that the Magento team most likely changed their minds a few times and took different paths to solving the same problems. It is thus perplexing at times, even for developers, to try and find what they need to modify so as to translate a page on the frontend.

The Translations Manager extension is far from a one-size-fits-all solution in that respect. It eases the process of translating parts of the site, but not all. This introductory section will try to map the different aspects of translating a Magento extension and explain what our extension is here to solve. By the end of this section it'll be clear that a one-size-fits-all solution means replacing a significant portion of Magento's core and isn't either viable nor necessary.

Magento's Internationalization System

The above title is a bit misleading, as it implies that such a concrete and identifiable system actually exist. In fact, Magento uses several methods for translating texts and a site administrator would have to go to different places to translate different texts. Below is a description of those places.

The Principles of Magento Translations

Whenever working with translations, it's important to understand the underlying concept of how Magento translates stuff. In essence, all there is to it is that each string (a piece of text) has a translated string associated with it. This is the one thing that's important to figure out, as it isn't really as intuitive as it sounds. Whenever Magento encounters a string on the website, it checks whether or not it has a translation for it, with regard to the currently selected locale. There aren't any identifiers by which Magento can tell one string apart from another, other than the actual text. This means that if, for some reason, a single string needs to be translated differently at different places, you're in for a treat.

There is an exception to the principle just stated, and that is module scopes. Magento, as an application, is build from lego pieces called modules. Each module serves a different purpose: one for the catalog, another for the checkout process, yet another for generic page rendring, and so on. When Magento is parsing a template file that is associated with the checkout module, it'll only look at translations that are associated with the checkout module, as well as generic translations not associated with any module at all. This creates a perculiar situation with regard to the above stated scenario. If the identical strings you're trying to translate differently exist under different modules, you'll easily translate them differently within their modules' corresponding translations files (see more on that below). If the strings exist within the same module scope, you'll have to trick Magento into thinking that they are different. The easy way to do that is stating different original strings, with identical English translations and different translations for whatever language you're working on.

The above passage states that the identifiers by which Magento recognizes translatable strings are the strings themselves (the actual text). In addition to the peculiar situation of trying to differentiate identical strings at different locations, this also prevents us from trying to use translations at a higher resolution than that of treating the whole string as a single unit. When Magento looks for a string it can translate, it treats the entire block of text contained within Magento's translation function as a whole. This means that you can't just try to translate a single word, or translate some generic phrase and expect Magento to modify any variation of it accordingly. Each string has to be translated on its own, even if there's just a single letter differentiating it from another.

So to conclude, Magento's translations are built out of a string, a translated string, and an optional module scope. To that, we'll have to add the locale and store views, but more on that will follow near the end of this section.

The Locale Folder

The main location to look for when in need of translating texts in Magento is the locale folder located under the app folder that's on the root of your Magento installation. In there, you'll find a folder for each language code. Inside each language folder, you'll find a bunch of CSV files, where each one corresponds to a module that's part of the Magento application. Finding out which module file you should be looking at is a bit complex, and sometimes you'll find that texts are located in unintuitive locations. The easy way to figure this out is trial and error. The harder but more exact way is to find the piece of code that calls on the translation function and see which module it belongs to.

The Theme Translation Files

Usually, theme files (*.phtml) belong to some module, and when they call on a translation function, the text could be found in the corresponding module's CSV file. However, as some theme files might have texts that are unrelated to any module, each theme can have a locale folder of its own. This can be found under:

root_directory/app/design/package/theme/locale

Under the locale folder you'll again encounter a folder for each locale code, and inside those folders would be a single 'translate.csv' file holding all the translated texts for that theme. If you're looking into translating a text of your own, and would like that translation to apply everywhere and not just in some given module's context, this would be the place to put it in.

The Administration Panel

Module and theme CSV files are used to translate module and theme strings, which would include most of the default stuff you'll see on the frontend after installing Magento. However, sometimes you'll need to translate data you've created yourself, such as CMS pages, products, attribute labels, etc. Some, but not all, of these types of data are easily translatable using the administration panel's user interface. A store switcher dropdown menu allows you to select a scope and you can choose to either type in specific values or use the default store view value instead.

You'll soon find that a significant portion of those types of data are not that easily translatable. The list includes CMS pages, category names, and the default texts defined under the configuration menu, such as the welcome message, or even the store's name. Usually, translating these strings requires workarounds. If you're interested in an easier solution, you might want to take a look at our free custom variables anywhere extension.

Something to note with regard to translating strings within the administration panel, is that this system is completely disconnected from the translations mechanisms described through the rest of this section. When translating an attribute's label within the admin panel, you're not creating a pair of original and translated strings that would apply in other places as well. What you're doing is configuring a specific attribute's label to have a specificly different value for a certain store view. As the two systems are disconnected, there's no point in trying to translate these strings using inline translations or CSV files.

Inline Translations

Magento offers a built-in feature that allows site administrators to translate texts directly on the frontend. This doesn't cover all the translatable strings, and might create a security problem if not used with care, but overall this is a very helpful feature. Inline translations, like any other kind in Magento, are identified by the string and the module scope. This means that even when using inline translations, you should take into consideration that changes made at one location can affect others where identical strings appear. Note, though, that inline translations are always module-specific so the opposite case applies as well - you might find that translating a string in one location doesn't affect identical strings on other areas of the same store.

A note should be made to the fact that translated strings saved using this method can only be controlled later on using the same method. When saving a translation using Magento's inline translations feature, you're saving a value onto an area of the database that takes precedence over CSV file translations. If you'll ever try looking for a string that was saved using this technique, you won't find it under any CSV file. If you'll try to add it to one, you'll soon find out that this has no influence. The two ways of gaining control over these translations are either finding them on the frontend with the inline translations feature turned on (which might be risky for live websites) or locating the database row and manipulating it by hand.

There's an interesting feature worth noting with regard to inline translations. These translations can be either store view specific, or apply to the entire locale. This is only relavant in case multiple store views use the same langauge but still might require different translations. This kind of distinction cannot be made within translation CSV files. The following sub-section might help explain this issue.

Stores, Store Views, and Languages

The Magento application allows a multiplicity of variations for different needs. Separate websites under the same installation might help with administrating separate businesses without additional IT overhead. For managing different catalogs, or a different pricing under the same catalog, different variations of multiple websites or multiple store under a single website might apply. When translating a single store to several langauges, the last level of the hierarchy - store views - is usually the one used.

Each store view is defined a certain locale. This is done in the configuration menu, under General/General/Locale Options/Locale. Locales are codes combining a language code and a country code. Once configured, the store view options will be available to the store's visitors through a select box on the top right corner of the website. Magento will automatically translate the page according to the locale defined for the store view that the visitor chose.

This creates a bit of complexity. Each translation is defined with association to either a store view or a locale. The module and theme CSV files, as well as the translations defined under the admin panel, are per-locale. These can't be defined with different values for different store views that use the same locale. This is, of course, a rather far fetched use case, but not completely out of question. In fact, it's so much not out of question that Magento enables that option percisely within inline translations. When using inline translations, you're given a choice whether or not to make the translation store view specific. In case it isn't, a administrative store view is used, and the translation would apply under any store view that uses the locale that was specified. When choosing to make the translation store view specific, both the locale and the store view are defined for that translation, so that it only applies under both conditions.

Translation String Parameters

At times, you'll find that strings you're trying to translate contain marks such as '%s' or '%d'. These are PHP patterns used to replace parts of static strings with dynamic content, and are used in many places in the Magento application. Understanding what those parameters will be replaced with requires either looking up the actual code or figuring things out from context. Note that parameters are used to serve dynamic content, but could also be used in different places where the same string is used, but with a different kind of dynamic content. That is to say, that looking up the code so as to find what is placed in a parameter really requires looking up every instance of code in which that string is used.

The Translations Manager Extension

It is the problematic architectural design of Magento's core to blame for the fact that such a long introduction is necessary so as to be able to state what the Translations Manager extension is, and what it is not. The Translations Manager extension replaces the module and theme CSV files, as well as the area in the database where inline translations are saved. The main ideas behind its construction are:

1. To provide a single location in which translation data is reviewed and manipulated.

2. To prevent duplications of translation definitions, that create scenarios where site administrators can't figure out why a string isn't being translated properly.

3. To allow non-programmers the use of the powerful translation parameters through a simple UI.

4. To allow site administrators to scan specific pages for translatable strings and then filter the translations list by page URLs.

5. And to replace the extensive parsing of CSV data with the more efficient use of database queries.

With these ideas in mind, we've created an administrative interface that allows site maintainers to control everything from a single location. Replacing Magento's mechanisms allowed us to add more functionality, and especially more control and ease-of-use for end users.

Following is the full list of features we've included:

1. All CSV files are scanned once during the initial setup and then ignored by the system altogether. This removes the complications of searching for a text in multiple files until something works, as well as the encoding problems prevalent when saving CSV files for Magento. At any time when new CSV files are added or changed, they can be re-scanned with a click of a button. Re-scanning can be done for the whole locale folder or for specific files only.

2. With the CSV files out of the way, defining module scopes doesn't require moving data from one file to another. A select box allows the user to choose which module scope this translation would apply to. This also means that selecting a scope isn't mandatory any more. If a translation is defined without a scope, it'll apply anywhere. If the same string is assigned a different translation for a specific scope, the default value will only be used where no specific values exist.

3. Strings can now be set to a disabled state and re-enabled when needed, either in bulk or one by one.

4. When editing strings, the translations can be set for each langauge by switching store view scopes. Everything is accomplished from the same page.

5. Translation parameters are edited from a simple to use interface. You can add new parameters, remove existing ones, and set whether to use a dynamic value from Magento's code (as is by default) or replace those values with your own. Parameter values could be either static texts or Magento's custom variables. You can decide on the order assigned to parameters, whether they are your own or the ones hardcoded within the translation function.

6. Inline translations work as before, only that the values are stored in the same place as all other translations and no duplicate entries are ever created. The results of working with inline translations can be reviewed and changed from the same administrative interface used for any other kind of translations.

7. The main administrative interface for managing translations allow sorting and filtering by: string, translated string, module scope, store views to which the string was already translated to, as well as the status (enabled or disabled).

8. To find out which store views a string was already translated into and which ones it weren't, two methods are available. First, a column in the translations table displays the list of translated store views for each row. Second, the management area has a store scope filter that switches the translated string column to different languages, according to the store view that was chosen. This way, it's possible to easily overview the list and see where translations are missing.

9. For every string, translations are per store view, and not per locale. This allows simplifies things and allows different translations for separate store views that use the same locale.

10. A simple export button allows downloading the data in the table in either CSV or XML format. The CSV format contains all the data from the table, but could easily be transformed to Magento's translation CSV format if necessary.

11. Bulk actions are available for deleting the strings, deleting just their translations in the current store view scope, and changing statuses (enabled/disabled).

12. A 'Scan A Page' section is available, through which one can type in a URL path for a page in the store and page will be scanned for translatable strings. On the management page, a filter by URL path option is available. Also, when editing specific strings, a table is available listing all the pages in which this string found when scanning. For *.phtml files, a file path and offset are also available if one wishes to find the actual file and modify it.

13. Configuration options are available for not bypassing module/theme CSV file parsing. This is helpful for live websites that wouldn't want to take down the website until the all CSV data is migrated to the new extension, or for any other specific scenario that requires it. Any data defined within the extension's management area takes precedence over the CSV files.

 

Having said that, it's just as important to state what the Translations Manager extension is not. For instance, this extension is not meant to optimize processing times involved in Magento's translation process and does not come with any guarantee as to page load speed. Just as well, the Translations Manager doesn't replace any of the administration panel's translation features. Every place on the admin panel where one has the option of translating some value is a separate database table and each employs a slightly different mechanism, based on specific needs. In effect, the admin panel have no translation system of its own and thus one can't be replaced. And so, attribute labels are translated where configured, and so are attribute options. Product attribute values have a separate mechanism for translating values and are configured under the product edit form. Same holds for custom variables, shopping cart price rules, and every other part of the administration panel. Replacing all these sections with custom code would require replacing most of Magento's core and isn't a viable option.

Installation

The Translations Manager extension was built on top of Magento CE ver. 1.7.0.0 and was not backward tested in any way. Before installing this extension, please contact us to make sure we've tested the extension on the version of Magento you run.

 

Installing the extension is accomplished the same way as any other Magento extension:

1. Go to Configuration->Magento Connect->Magento Connect Manager.

2. Re-type your administrative credentials.

3. Under 'Direct Package File Upload', choose the file you've received from us, click upload, and commit changes.

 

If any error messages come up before or after commiting the changes, copy them somewhere so we'll be able to assist later on. Undoing changes yourself at this stage could be quite risky, but if you're a technical person and some problem requires a violent removal of the extension, here are the steps to take:

1. In the database, drop the tables named: 'translator_string', 'translator_translation', 'translator_cache' and 'translator_path'.

2. Under the core_resource table, delete the row that contains the code: 'translator_setup'.

3. Remove the folder at: app/code/local/Wheelbarrow/Translator

4. Remove the file: app/etc/modules/Wheelbarrow_translator.xml

5. Remove the folder: js/wheelbarrow

6. Reset Magento's cache and session.

 

Please note that this extension will disable every CSV file and inline translation you've defined. Only once the initial setup described below is complete, all former functionality will be restored. For assistance with the setup, especially if you're deploying to a live website (which we deplore you not to attempt), please contact us and discuss the details.

The Translations Manager extension isn't a minor change. It'll change the entire operation of Magento's core with regard to translations, which is a heavy-duty assignment. Take this installation seriously, test on a separate environment first. If you're running a very weak server you should take that into account and contact us to discuss the process. Trying the extension out by just clicking on buttons without reading the 'initial setup' section could have devestating consequences. With that said, a few perliminary things to check for would be a decent value for the memory_limit parameter under php.ini (128M is advised), and a large enough value for the max_allowed_packets parameter of your mysqld service (at least 1000000).

Initial Setup

The Translations Manager extension comes without any knowledge of your system's configured translation data. Once installation is complete, you need to start importing the data from Magento's default locations onto this extension's database table. Doing so only requires working within the administration menu, and there's no real need in understands the intricacies of Magento's core or handling the database itself. However, the importing process is resource-intensive, a lot more than regular website operations. It is therefore recommended that you test this first on a non-production enviroment. Furthermore, two variations are supplied for importing module CSV files: one for slower servers and one for faster ones. The decision is yours to make, but do it with care.

A note should be made, that until the initial setup is complete, your store's current translations are not active. The Translations Manager extension disables the Magento's use of CSV files by default. If you'd like the store to keep working with the CSV files until you'll finish importing everything, you'll need to configure the extension to not override these files. Doing so can be done through the configuration menu, under: Configuration->Wheelbarrow->Translations Manager->Options. You'll find two fields for choosing whether or not to override current theme and/or module translation files. If you choose 'No' for both and save, you'll regain the current behavior of your store for the time being. Note that in the long run, if yuo choose to work with both CSV files and the Translations Manager, the latter will take precedence in any case of conflicting values.

All the import work is done from a single admin page, found through the administration panel's menu, under: Internationalization->Sync Resources. If at any point in the future you'll need to add more data from CSV files to the Translations Manager, this will be the place to go. However, use it with care, as importing data could override existing values within the Translations Manager, in any case of conflicting values.

 

Each import operation overrides any previous records already in the Translations Manager, in any case of duplicate entries. You should note that if you've done work using the Translation Manager and then imported conflicting values, any imported value will override previous ones, so take care. The order in which you follow these instructions is important, in this respect. The order in which Magento loads translation data during page rendering is: (1) module files, (2) theme files and (3) database. So following the order described here would imitate the way values are overriden within Magento and thus maintain the current behavior of the website.

 

Once you're done importing data, you can start using the extension. All the data you've imported is available under the 'Manage Translations' area.

Importing Module Translations

Module translation files are the most extensive, and thus the most complex to import. Two options are available in this case. One, in which everything is imported in one 'go', and another, in which you import specific files manually. The first option is not advised in any way. The more data you import, the slower the server will be to render pages for website visitors. Importing everything just doesn't make since, so if possible, follow the second path and import only the files you need. If for some reason, you would like to take the first (un-advised) path, note that its description contains a little more technical jargon.

1.1 The un-advised path: For those running fast servers, especially those who run on localhost at the time of the installation, the simplest course of action would be the 'Import CSV Data From Modules' button. An avarage language would have ~7,000 translation records, so if you have just one extra language on top of the English default files, you'll be importing ~14,000 records into the database. This is done in batches so that the server won't crash, but a weak server (such as an AWS EC2 Micro instance, and even a small instance) won't be able to handle repeated calls even when each call is just for a small amount of data. The default batch is for 30 records and you can play with that through the configuration menu. Running on localhost, this usually works like charm, but be advised that the process does take quite a bit of time.

1.2 Importing individual files: To begin the process, click the 'Scan For Module CSV Files' button. The table below will be filled with all available combinations of locales and modules. Each row in the table corresponds to a file on your server, and you can choose which files are relavant to you. For example, any file for the en_US locale is a translation from the same language and is thus not really necessary. For each such combination that you do wish to import into the Translations Manager extension, just click the row in the table. A server processing stage will begin, parsing the translation records in batches of 30 each time. Just above the top of the table, you'll see a status update on the process. Once it's complete, the status update will notify that it's done processing and you'll be able to continue executing other operations on the screen. A 'Status' row keeps track of which files were imported and which were not, but because CSV files can change, the statuses are all reset every time the 'Scan For Module CSV Files' button is clicked.

In any of the scenarios described above, if the browser is stuck and the operation is halted without coming to an end, the unprocessed batches are stil waiting in the extension's cache. Once you initiate another CSV processing job, these batches will be processed as well. If you'd like a clean start (especially if you accidentally clicked the button that tries to import all the module files together), just click the 'Clear Sync Queue' button and the batches will be removed from the cache. You should also note that the file status changes to 'In Sync' whenever you initiate its import and without regard to whether or not the process finished successfully. The only real way to be sure a string was imported is to look for it in the traslations management screen.

Importing Theme CSV Files

Because theme CSV files are usually relatively small and not in abundance, importing them is done in one blow with the 'Import CSV Data From Themes' button. Just like with CSV files, you'll soon see a status update just above the table and once the process is done, the browser will be released for further operations. Only the theme files used within the current theme hierarchy are imported, so there's no point in importing them individually as was the case with the module files.

Importing Inline Translations

To use inline translations you've previously defined, you'll need to import Magento's translations table from the database into the Translations Manager. To do that, all you need to do is click 'Migrate Data From Database' button. Note that if you installed Magento with sample data, about 40 records with attribute labels were defined in that table and they would also be imported to the Translations Manager.

General Usage

Note that translation data is cached by default in Magento. While working on translations, it is advised to temporarily disable this type of caching under the 'Cache Management' screen. If this isn't possible, you'll need to refresh that cache each time to view your changes on the website. You should also note that leaving this cache disabled on a live store running the Translations Manager isn't at all advised. This extension relies on the use of caching for decent operation and would make page load times ~4 times slower if were cache is disabled.

A quick note on string quoting rules is due. Strings and translated strings pass a process called quoting before being saved to the database. During this process, special characters (quote, double-quote and backslashes) are prefixed with backslashes so as not to create confusion. This process should be completely transparent to the end-user, and indeed, the strings you'll review on the administration panel will not have backslash prefixes. Awareness to this process is important because it unavoidably modifies backslashes in the original strings. The rule of thumb is that if you'd like a backslash within your string, use a sequence of two backslashes one after the other. You'll notice that once you save the string, one of the slashes disapears. Altogether it is advisable to avoid using backslashes and not have to deal with these coding problems, but if you require a backslash within the actual string, this would be the way to achieve this. Using quotes and double quotes, on the other hand, should be possible without any complications and without needing to prefix anything yourself.

Find and Review Translations

The main panel of the Translations Manager can be found through the administration panel's menu under Internationalization->Manage Translations. There, you'll find a grid table containing all the translations you've already imported. The 'Visible Scope' field at the top allows you to view the same strings translated to different locales based on the store view selected. The actual table presents the strings and their translations (with regard to the current store view scope), the module scope to which the translation applies, a list of store views to which this string was already translated and the status of the translation.

Filtering and sorting are standard Magento admin features and these functionalities are accoplished here the same way as in any other panel of the admin area. At the top of the table, a bulk action field is available for selected rows. It enables deleting a selection of strings, deleting just their translations (according to the selected store view scope) and changing statuses from enabled to disabled and vice versa. Above the table's area there an 'Add a New String' button whose purpose is most likely self exaplanatory.

Add / Edit Translations

Clicking on a string's row opens up the string edit form. From there, you're able to change any of that string's attributes. Note that changing the original string means in essence that you're creating a new item. If another item already exist with that exact combination of string and module scope, that other item will be replaced with that being saved now. If, when creating or editing an item, you choose 'None' under the module field, that item will be used anywhere in the site (except for where more specific definitions exist).

If you're adding or removing parameters from the string itself, you can control the value and order of the parameters using the table below that field. Click 'Add Another Parameter' to add a new row. Using values from the code requires checking the box under the 'Value from Code' column. The same column contains a text field in which to place the position of the parameter you'd like to use. This is different from setting the position of the parameter being configured. The combination of these two fields can allow a site administrator to play around with flipping the position of existing parameters within that string. In any case, if a value is not defined (either from code or a custom one through the 'Value' column) then the value is extracted from the code as it would by default. This means you can always leave this table empty in case you have no need in playing around with the parameters, which is usually the case. Note that any changes done to parameters has a minor effect on performance. There will always be a tradeoff between more administrative power and a more optimized code, and this is a textbook example of that principle. Defining a parameter value yourself is usually not necessary, but you have the option of typing in a static value into that field. You could also use custom varibles in this field, using the standard syntax: {{customVar code=YOURCODE}}. This could allow you to use the same values for a group of strings and only change that value from one single location, saving both time and possible human error in the process.

To translate the same string into different languages, you can switch store views on the top left corner of the screen, modify the translation field, and use the 'save and continue edit' button to preserve changes before switching scopes again. Any unsaved changes when changing store view scopes will be discarded. To remove certain translations from existing items, just hit the 'Delete Translation' button when in the intended store view scope. The translation will disappear and this store view will revert to the value defined for the default store view (or the original string, if no default store view value was defined).

Scan Pages For Translations

A cool feature of the Translations Manager is scanning pages in the store for translatable strings. To do that, in the main menu, go to Internationalization->Scan a Page. You'll be led to a simple form with one big text box and a button. Paste the complete address of the store page you'd like to scan (with http, https, or the protocol-independant syntax - '//') into the text field and hit the button. If your browser is blocking pop-up windows then nothing will happen, so make sure you allow the browser to open pop up windows. If that's the case, a window or tab will open up with the page you've requested. It'll have a parameter attached to the URL which tells the system to scan the page being loaded. As the page loads, you're able to see which blocks are scanned and which phtml files are opened, through the status updates on the form. When the status update notify that it's done, you can close the extra window/tab and go back to the 'Manage Translations' area. All the translatable strings found on that page are now added to the table. You can filter by URL path using the text box right above the table to see all of them. Just type in the same address or a part of it and click 'Search'.

If you'll open one of the strings found in the scan, the 'Associated Paths' tab found to the right of the edit form will contain the list of paths this string was found in. In any case where the string was found in a .phtml file and not a .php file, the exact path of the file and the offset it was found in would be specified as well. Note, that if you'll try to change the original string on such an item, all paths data will be removed, as it is no longer really the same string. This was done to prevent confusion as to what really happens when changing the value of a string. It's not that every location where that string is called is also modified, it's just the translation definition that now applies to a modified text.