Recommended Addon

[Goofy]

This topic is dedicated to discussion and maybe realization of the idea as initially displayed here
http://babelwiki.babelzilla.org/index.php?...testing_machine

I am glad to say that this dream may have a little chance to come to reality, since I received encouraging feedback from Davide Ficano (dafi) and Martijn Kooij (Captain Caveman) not to mention Fenian himself. Of course it is certainly a huge work and I don't expect it can be done as quickly as an extension.

I paste here some significant observations. Everyone is welcome to comment, suggest, and why not, contribute


From dafi

what you say can be easily done (at least 90%) directly on BZ using the scripts I've written to parse install.rdf, it is easy to add support for chrome.manifest.
I can try to develop a webapp, I'm not sure developing an extension is the correct way, in this case a server side application sound better.

From Captain Caveman

Practically:
In my oh so humble opinion this would be a bad idea. Because it puts the responsibility of a good localization away from the localizer into the hands of the developer. So if at all, it would seem better to create an automated way for a localizer to test his/her own work. And since a localizer should already feel responsible to test his own work, I don't think this process can be made any easier for him/her.

Theoretically:
- I think a prerequisite must be some extension being installed in the testing Firefox to make a couple of things easier (installing addons, reporting problems)
- An application (either web/client) could process the xpi, and parse the required information as needed, that should as Davide said, not be the hard part.
- The application could start Firefox with a command parameter indicating the extension to be installed, our pre installed helper extension would then capture this parameter and install the extension indicated, writes an install succeeded message in a logfile the main application monitors, and closes Firefox.
- Next the application would start Firefox with the command parameter -UILocale and so setting the correct locale to test.
- Our helper extension could check for errors in the console (Firefox 3 only?), report these errors or a test succeeded message in the logfile, and closes Firefox again.
- The main application (which was monitoring the logfile) processes the result, and continues accordingly.

But that quite a lot of work I think...

[DaveG]

Writing an extension to sanity check another extension's locale files shouldn't be too hard. Automating the setup of each sounds a bit overkill though. I think that it could test the whole set at once and then the users could check the actual showing of each locale themselves.

While I do agree that the server-side scripts should be doing some of this, an extension makes sense for translators to test their addition of their locale for their own purposes.

[Goofy]

I think that it could test the whole set at once and then the users could check the actual showing of each locale themselves.

Do you mean smthg like: all locales are tested in one pass, and a report is done (saying e.g "en-US -- ok it-IT--failed ...)
mmmh if every locale is tested, the app must restart every time with an appropriate general.useragent.locale, right ?

While I do agree that the server-side scripts should be doing some of this, an extension makes sense for translators to test their addition of their locale for their own purposes.

Sure extension translators will always appreciate some automatic checking. But my idea was more to be useful to developers at the moment when they update/upload their new version on AMO with a bunch of locales, or moreover it could be used by AMO reviewers and spare their time.

[DaveG]

Do you mean smthg like: all locales are tested in one pass, and a report is done (saying e.g "en-US -- ok it-IT--failed ...)

Yeah.

mmmh if every locale is tested, the app must restart every time with an appropriate general.useragent.locale, right ?

No. I'm thinking maybe just load the files manually and check to make sure everything is where it should be and no entities are missing. You'd still have to set the correct pref and check yourself to be sure. (i.e. with quick locale switcher)

Sure extension translators will always appreciate some automatic checking. But my idea was more to be useful to developers at the moment when they update/upload their new version on AMO with a bunch of locales, or moreover it coyuld be used by AMO reviewers and spare their time.

Yes, that sounds like a good idea.

[Goofy]

No. I'm thinking maybe just load the files manually and check to make sure everything is where it should be and no entities are missing. You'd still have to set the correct pref and check yourself to be sure. (i.e. with quick locale switcher)

Sure Quick Locale Switcher is a very useful tool, but I was dreaming of a more complete automation, with almost no "human" intervention: once fed with the xpi, the "testing machine" would just know which lang pref to set and cycle through locales, restarting Fx as many times as necessary... (I know, it is big challenge here)

[DaveG]

Sure Quick Locale Switcher is a very useful tool, but I was dreaming of a more complete automation, with almost no "human" intervention: once fed with the xpi, the "testing machine" would just know which lang pref to set and cycle through locales, restarting Fx as many times as necessary... (I know, it is big challenge here)

Nothing automated will be able to tell the user if something just looks wrong. If what you're suggesting is that it just load each up for the user to check, thus saving them a step, then that makes sense.

[Goofy]

Nothing automated will be able to tell the user if something just looks wrong. If what you're suggesting is that it just load each up for the user to check, thus saving them a step, then that makes sense.

hey it is not here a question of "looking wrong" (you are right, nothing can automatize translation proofreading), but to detect xul errors on install/start or on trigerring the options.

[Goofy]

Hey I also have an answer and suggestions from Axel Hecht

my question was

I discovered this set of scripts you created
> http://mxr.mozilla.org/mozilla/source/test...s/l10n/scripts/
>
> Coul not these tools be used to check extension locales as well, or how
> could they be modified to have some kind of automatic checking?

His answer:

Yes, and to some extent they're designed to do so. I like your idea to
look at the chrome.manifest to gather locales.

The code is currently on one of my hg repos,
http://hg.mozilla.org/users/axel_mozilla.com/tooling/, and some of my
peers are rewriting the backend to be more capable in a project called
silme. http://hg.mozilla.org/users/zbraniecki_mozilla.com/silme/ has the
code, though unstable still.

Those libs are good for some source code checks. They're not catching
all kinds of errors yet, and might never completely do so.

Thus, adding some runtime testing on top of that might be a valuable
idea. To efficiently do so, I'd suggest to look at mozmill, a testing
environment developed by Mikeal Rogers and our QA folks. Read more about
that on Mikeal's blog, http://www.mikealrogers.com/. Somewhere beneath
you'd find https://wiki.mozilla.org/QA/TDAI/Gristmill/Mozmill_Tutorial
explaining how to write tests, which the extension authors should
provide, at least beyond a certain point.

HTH

Axel