• Do not register here on develop.twiki.org, login with your twiki.org account.
• Use View topic Item7848 for generic doc work for TWiki-6.1.1. Use View topic Item7851 for doc work on extensions that are not part of a release. More... Close
• Anything you create or change in standard webs (Main, TWiki, Sandbox etc) will be automatically reverted on every SVN update.
Does this site look broken?. Use the LitterTray web for test cases.

Item4419: I18N: Non latin characters in field names break forms

Item Form Data

AppliesTo: Component: Priority: CurrentState: WaitingFor: TargetRelease ReleasedIn
Engine I18N Normal Confirmed   n/a  

Edit Form Data

Reported By:
Applies To:
Current State:
Waiting For:
Target Release:
Released In:


My TWiki configured by use utf8 encoding.

I define form template:

Name Type Size Values Tooltip message Attributes
Аннотация text 85     M

New topic body:

%M ETA:TOPICINFO{author="FalKo" date="1185536256" format="1.1" version="1.1"}%

-- Main.FalKo - 27 Jul 2007

%M ETA:FORM{name="ItemTemplate"}%
%M ETA:FIELD{name="A" attributes="M" title="Aннотация" value="123"}%
%M ETA:PREFERENCE{name="VIEW_TEMPLATE" title="VIEW_TEMPLATE" type="Set" value="ItemView"}%

I try save topic which use this form template, but FORMFIELD meta data is not saved in my new topic.

Then I try mixed whith сyrillic characters and latin like this: "FАннотация". When click submit form I see error report "required fileld "FАннотация" can't be empty".

I try replace fist letter А-cyrillic to A-latin. It's right, field saved correctly, but name attribute is not "Summary", but latin character 'A' from mixed "Aннотация".

Why name attribute is name of, but not link?

-- TWiki:Main/AndreyTkachenko - 27 Jul 2007

I changed a encoding to KOI8-R, but it did not decide a problem frown

-- TWiki:Main.AndreyTkachenko - 27 Jul 2007

Changed attribution to I18N (internationalisation) and headline.


I said that this problem does not depend on the code of Utf-8.

it is correct it will be to say that not Latin characters are unright processed

-- TWiki:Main.AndreyTkachenko - 08 Aug 2007

Andrey, please contact TWiki:Main.RichardDonkin directly on this issue, I'm afraid he is really our only I18N ressource currently.

-- TWiki:Main.SteffenPoulsen - 20 Aug 2007

Possibly related to Bugs:Item2032.

-- TWiki:Main.SteffenPoulsen - 20 Aug 2007

It's not related to Bugs:Item2032. And not UTF-8 character!!!

-- TWiki:Main.AndreyTkachenko - 21 Aug 2007

You stated that you were using utf8 encoding, this confused me. Sorry for that, I hope you will forgive my ignorance.

-- TWiki:Main.SteffenPoulsen - 21 Aug 2007

Andrey, this is not the same as Bugs:Item2032 - however, if you had read that page you'd have seen a link to TWiki:Codev.InternationalCharactersInFormFields, which is the same issue. That bug is complex to fix for all languages/scripts, but attached to that page is a patch that may help you - just a matter of commenting out one line in the Form.pm file's _cleanField routine. Note that this patch is for a much older version of TWiki, so it probably won't apply directly - so just comment out the line that looks like:

   $text =~ s/[^A-Za-z0-9_\.]//go;

In more recent versions of TWiki, the routine is called fieldTitle2FieldName, and has the same function (see SVNget:lib/TWiki/Form.pm). Unfortunately, in the refactoring someone managed to remove the I18N TODO comment which highlighted this issue.

One side effect of this change would be that your field titles would be used unchanged as field names. I'm not sure why this is a bad thing, but normally TWiki uses field names (i.e. internally) that are the field title with all non-alphanumeric characters removed. So any form field data created with this line commented out may not be accessible if you later install a version of TWiki where this line has not been commented-out - so be careful!

This bug is independent of language and character set incidentally, but is most serious for languages that don't use Latin character sets.

To anyone else reading this page - the patch suggested here and linked from TWiki:Codev.InternationalCharactersInFormFields was rejected by PeterThoeny so is not generally a great idea. Also, it needs more work to cleanly support East Asian languages such as Chinese and Japanese, though it does work for those too.

Next Action: other TWiki developers to comment on whether it's OK to make field name same as field title, including upward compatibility issues etc. Some move in this direction is really required to make this code work at all for I18N.

-- TWiki:Main.RichardDonkin - 23 Aug 2007 Can some other developers comment on whether my proposed Next Action above is OK? May break upward compatibility, suggestions welcome. -- RichardDonkin - 11 Sep 2007

Summary I18N: Non latin characters in field names break forms
ReportedBy TWiki:Main.AndreyTkachenko
Codebase 4.0.0, 4.0.1, 4.0.2, 4.0.3, 4.0.4, 4.0.5, 4.1.0, 4.1.1, 4.1.2
SVN Range TWiki-4.1.2, Thu, 19 Jul 2007, build 14438
AppliesTo Engine
Component I18N
Priority Normal
CurrentState Confirmed


TargetRelease n/a

Edit | Attach | Watch | Print version | History: r11 < r10 < r9 < r8 < r7 | Backlinks | Raw View |  Raw edit | More topic actions
Topic revision: r11 - 2007-09-11 - RichardDonkin
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback