Hey folks,

Here’s what I’ll be working on in the coming weeks, and why it may look like not much is happening Form Tools-wise… All the following features relate to Open Translate, Form Tools’ translation management script.

1. Google Translate

In the last couple of years Google has come out a number of extremely cool stuff relating for translation, including the Google Translate API which allows developers to translate content via their own scripts. For the foreseeable future, automated translation of text is never going to be as solid as that done by a real translator, but it’s a good start. My plan is integrate Google Translate API with Open Translate to provide a base translation of all languages that they offer. These translations will then be open for other translators to review and improve.

Depending on the quality of the translations, this should significantly reduce the amount of work for translators as well as provide an initial – albeit bug-laden – translation file for many languages that haven’t been tackled yet. These languages will then be bundled with the main download file so that people can choose their language right in the installation script. To encourage better translations for all of us, I’ll be storing the Reliability Percentage (see: translations.formtools.org) of each translation in the corresponding translation file. Until that percentage is > 90% (or something), I’ll include a link in the admin UI to encourage non-English speaking users to help translate their language.

2. Data Versioning

So… this is a fun one! A little backstory first. I wrote Open Translate (opentranslate.org) a couple of years ago to properly handle community translations, and up until now it’s held up pretty well. But now that Form Tools 2 is out of Beta, I need to complete a feature that I spec’d out but never completed: versioning. The text that needs to be translated for Form Tools 2.0.0 is now FIXED, I won’t be modifying it. This is going to be the case for all “final” releases from now on. You’ll be able to download a translation file for whatever version you’re using. [I HATE WordPress's annoying "You're version is out of date! Upgrade now!" line that appears at the top of the screen every few weeks to persuade people to upgrade. I won't ever do that. You can stay on v2.0.0 for years to come if you want, and I won't bug you. Promise. :) ]

So anyway, the goal is to allow translators to be able to pick a version that they want to translate – whatever version of the script is relevant for them.

Now here’s the catch: I don’t want to duplicate any text + translations for each version (2.0.1, 2.0.2 etc). I want all of them to share the same text + translations as much as possible to minimize the amount of work done (one of the golden rules of software development is duplication == bad!) Also, if a user is running 2.1.8 and translates some text that’s been unchanged & untranslated since 2.0.0, I want that translation to be automatically added to ALL versions from 2.0.0 onwards.

As you can imagine, this is getting complicated! Neat problem, huh?

My original plan was to base Open Translate entirely on SVN (Subversion), which so that all translations and data were versioned. Subversion is just that: it’s a versioning tool that keeps a repository of all data and offers ways to slice and dice the information depending on what you need. In fact, this is the script that powers the Form Tools upgrade mechanism and helps generate the custom zipfiles containing whatever files have changed since your last version. It’s used on this site to keeps a giant database of all Form Tools files: themes, modules, API and Core.

Despite its many virtues, after much hemming and hawing I decided against using it for Open Translate. Partly for technical reasons that I won’t go into, but also partly because Open Translate would still have needed a duplicate database of the data and translations o allow for quick searching and access of the data via a web-based UI.

So…. long story is that I’ll now be adding in versioning support to Open Translate. I have it all sketched out and it’s a thing of beauty.

3. Module Translations

This is something I’ve been hopelessly delinquent about. Once the above two are complete I’ll be adding the option for modules to be translated as well. Right now all of them have a single, English language file. Weak!

And that’s all folks.

- Ben

The first draft of my company site is finished, so check it out! It’s all Ajax-driven right now, without any non-JS fallback, but in time I’ll add the static HTML option.

Right now the site is nothing much more than a catalog of my main scripts, but in time I’ll expand it to include my other scripts – as well as a location for future releases. I’ve been working on a invoice generation script which I will be releasing there in the next couple of months.

Next, I’ll be re-branding my other scripts under the company name.

Visit the Black Sheep Web Software website.

Just brushing up my résumé… a few more screenshots!

Today was my first day where I was finally settled in Saskatoon, so I was able to catch up on some emails and a little work. I think I’ll enjoy it here, albeit for a mere 9 months. I was undisturbed the entire day and even got a little contract work done. Things should be peaceful in this place.
But here’s the really exciting news…

Form Tools 1.5.0 Private Beta Released! (Language Packs)

Now available! Unlike previous Private Beta versions, I’m only making this available to translators who wish to volunteer to translate the UI. This version represents a major refactoring of the code to support multiple languages.

If you’d like to help translate, send me an email here. So far I’ve had people volunteer for Dutch, Portuguese, Chinese (Simplified and Traditional), Spanish, Italian, French, Farsi (Persian), German and Polish. A good start for testing the software.

Open Translate Beta Released!

I’ve finally opened the doors to a handful of translators who will use it to translate Form Tools. It’s VERY basic right now, and missing a front-end. This has been a loooong project, and rather than put it off even longer I decided to push it live.

It’s pretty cool, actually. Other than for fixing bugs, I could walk away from it if need be: all translations made through opentranslate.org are automatically zipped up and exported every hour to the Form Tools site for download by the Form Tools beta testers. In theory, the translation component of Form Tools will be entirely self-managed by the translators themselves. I can worry about other things!

A very satisfying day indeed.

In the upcoming month we’ll be releasing a new Private Beta of Form Tools featuring (drum roll, please)… multiple language support! All aspects of the Form Tools UI are now localized – display text, notifications, confirmations, error messages and even dates.

In preparation for this exciting event (well, I find it exciting!), if anyone would be interested in helping translate Form Tools to their native language – or even just help review existing translations – please drop me a line at [email protected] with your name and language you speak. Once things are ready I’ll set you up a user account and provide further instructions.

Andrew Jakubowski and I have moved Open Translate to a dedicated server, and we’re just putting the finishing touches on the translator pages.

Looking forward to hearing from you!

Boy this was a pain. Here’s the UNIX command for generating a .po file from all PHP files in a folder (and subfolders).
find ./ -iname “*.php” -exec xgettext -L PHP -j -o messages.po –keyword=_e {} \;Â

- Note: the messages.po file needs to exist first.
- Note: the –keyword=_e clause lets gettext know to parse strings within calls to the function named _e().

It still produces a bunch of warnings, but I gather they’re harmless.

I thought it was time to write another update about Open Translate. This time I’ll include a general overview of the project by way of introduction.

About Open Translate

Open Translate is a translation management system, especially geared towards open source / free projects. It provides a common interface for project managers and translators to import, translate, manage and export data.

The age-old problem with having anything translated is that you doesn’t always know when a translation is good – not unless you speak the language it’s being translated into. With projects that require translations into many languages the likelihood that you speak every one becomes increasingly unrealistic: how many languages do you speak?

Open Translate solves this problem. It employs a review-based mechanism to authenticate each translation. Each translation must reach a “review threshold” before it’s approved; multiple translators must review each translation and mark it as approved before the translation is completed and closed.

Projects can be either public or private. Open Source projects would generally be set to public: anyone could sign up to help translate (whether or not a person may join the project is either through a configuration setting: automatic or explicit approval by a Project Manager). Companies which have their own in-house translators would set their projects to “private”: the content wouldn’t be visible to anyone but themselves, and only their own translators would be able to work on them.

Behind the scenes, statistics are compiled for each translator. This will be valuable for companies needing to gauge the experience and quality of a translator’s work. For example, if Greenpeace set up a new project and needed translations into German, they could review a list of all English-German translators and contact the people with the highest translation reliability, number of translations and reviews. Translation reliability is calculated by reviews made by the translator’s peers.

All projects have a built-in “versioning”. At any point, you can make copies of the entire project for separate translation. For example, imagine you had your project translated last year, but this year you need the same thing translated again – except a few minor modifications to the original text. Instead of re-translating everything, you can simply create a new version, make the changes to the original text, and then open the new changes for translation. This has the advantage of keeping both versions entirely distinct and keeps the translation work to a minimum.

The Game Plan

The goal of Open Translate was to create a self-managing system for the entire translation project – and it’s getting pretty close. The site will offer nightly exports of the translations via FTP (or manual download) to whatever server you need. The initial release will be PHP-centric: data will only be imported and exported in PHP format. But this will be expanded to XML, Excel, CSV, by generating a populated MySQL table and, of course, the standardized .po and .mo formatted files generated by GNU gettext. Once that is completed, we’ll be focusing on simplifying the process of extracting text for translation.

A Few More Screenshots

Here’s a few more screenshots of some of the software components. There’s still nothing on the public site – these are all backend pages for translators and administrators. For the initial release I’m not so much concerned about appearance of the site than the functionality and robustness of the code.

Project Dashboard

This is the main project dashboard for administrators and project managers. It provides all the options available to manage your project from importing / exporting data, setting up categories, project versions, the project description and viewing / blacklisting translators associated with the project.
The icons, incidentally, are all LGPL. I had to dump the earlier icons in favour of these for licensing reasons (plus I like these better).


Statistics

The initial release will provide some simple statistics for seeing the state of each language translation: the total percentage translated and the percentage reliability. The second value is calculated based on the projects “review threshold” (i.e. the number of times you require a translation to be reviewed by distinct translators before it’s approved.

Administration: Manage Translations
This page gives administrators / project managers a top-level view of the status of each translation for each language. The green boxes signify whether the translation has been made (the first box), and how many times it has been reviewed other translators. The review number may be configured for each project.

Administration: Translation History
Administrators / Project Managers may view the entire translation history of a translation to see how it progresses. This page lets you manually override the default review queue and approve a translation if you feel it is satisfactory or trust the translator. This page is also handy to track down dud translators and blacklist them from your project.

Translator: List Translations
This page is for translators: it lists all translations in the system and provides a simple UI for changing what data they wish to see: by category, text size, whether it requires translation or review.
There is also the option to “bulk translate” and “bulk review”: translate or review multiple items on a single page.

Translators: Review Translation
A simple translation for review. This page lets the translator review an item. If they mark it as anything but “excellent”, they have to improve the translation.

This has been a fascinating project from day one. I’ve learned a lot about the deficiencies of PHP 5 for unicode – a major thorn in my side. PHP 6 promises to include full unicode support and boy will that be nice.

I’m shooting for having a fully functional Beta ready for the end of the month. Initially, as mentioned elsewhere, it will only be used for translating the Form Tools project and the Open Translate program itself – but once it’s been tested thoroughly I’ll open it up to the world.

A short update on the status of the Open Translate project.

Now that the Data Generator site is wrapped up, I’ve been easing my way back into Open Translate. I’d forgotten how cool it was! :-) Right now I’m working on the translator UI. The admin and project manager sections are mostly completed, but no doubt other stuff with crop up.

I pestered a couple of my co-workers for advice on some database aspects and to troubleshoot some business decisions and they were both bloody helpful. A rather nagging design aspect of the database kept slapping me in the face whenever I looked at it. It was a sod to code around and no matter how I sliced it, I didn’t find it intuitive. It’s now greatly improved. One of them, Andrew Jakubowski, will be assisting on version 2 of the software. Version 1 – which isn’t far off now – will really be just a proof of concept; I’ll only be using for translating Form Tools and Open Translate itself.

But, first comes a slew of freelance work: the Salvation Army project is plugging along (but I’ve made definite progress), there’s a site for an Art Gallery that’ll be starting soon and apparently I may be doing the Vancouver Pro Musica website. Should be interesting.

Just to prove I haven’t been sitting on my hands these last few months, here’s a few screenshots from the Open Translate script. It’s slowly coming together. I’ve spruced up the UI with some dynamic JS and AJAX for speed, but there’s still plenty to do. A good chunk of the backend is now complete: project managers can add and update project data, project versions, categories, translators, personal preferences, version settings, project settings and more. The full data export options are still limited, and for the initial version I’m only going to offer PHP format (since that’s what Form Tools requires). The entire UI supports bi-directionality and pulls all text from separate language files.

I’ll begin work on the Translators section soon, so I’ll be able to make it available to Form Tools translators. Pretty cool!

Just a few things to report, in no particular order:

• I spent today at the Vancouver PHP Conference. I saw a couple of good speeches: first, Rasmus Lerdorf – the inventor of PHP – presented the keynote speech. It covered rather a lot of ground, but he showed some rather interesting techniques in benchmarking and locating speed bottlenecks in PHP scripts. There was also a dry, but informative lecture by Andrei Zmievski on unicode support in PHP 6. Interesting, but I found myself wondering more about the current limitations in PHP 5.
• I finally got round to installing WordPress on the Form Tools site to manage the news, so now I’ll be posting all Form Tools-related news there instead of here.
• The translation project continues… I’m now able to manage projects, project managers, data and translator accounts. Next comes the ability for translators to view and actually translate the data. Should be a doddle (heavy sarcasm).
• I’ve started work on a couple of new contract jobs for Salvation Army and BP (Beyond Petroleum). The Sally Ann gig in particular is pretty big. I’m shooting to getting it done by the end of the month.