Tech: display bug

Subject: RE: Tech: display bug
From: Jack Campin
Date: 14 Feb 11 - 08:02 PM

When I do "View Source" in Firefox, what I see is perfectly normal HTML with no funny characters. The foreign letters only appear when Firefox tries to render it.

Example: this URL: http://www.hora.ro/popup/cobza.htm

It looks normal when I'm typing it in the message composition box. Once previewed or posted to a thread, it'll be Norwegified and the third slash will disappear. But if I copy the Norwegified URL into a text editor, it again looks normal: o-slash rather than Norwegian-o. The odd character is not formed by displacing the slash backwards over the o, it looks like an honest-to-god Norwegian letter (as does the Polish l).

The only add-on I have at the moment is BetterPrivacy, which deletes Flash cookies.

I think what I have to try next is make a copy of a Mudcat page with a character encoding declaration added in, and see if the problem still comes up.

Subject: RE: Tech: display bug
From: JohnInKansas
Date: 14 Feb 11 - 05:22 PM

Since the page code doesn't show what you see, if you copy the characters from your display - where you see them - and paste them into Word, with the cursor immediately on the right of the character "Alt-X" will toggle the display to show the HEX code number for the character.

The Hex number should let you verify what character the browser is trying to display. If Alt-X only changes "half the character" then the display is displaying the second one displaced back onto the first. If the whole thing changes, then your browser is somehow changing the char num for the character pair that mudcat sends you.

Suspect chars:

o    Hex 006F   Decimal 0111   o   o

O   Hex 004F   Decimal 0079   O   o

l    Hex 006C   Decimal 0108   l   l

L   Hex 004C   Decimal 0076   L   L

/    Hex 002F   Decimal 0047   /   /

0/   Hex 006F 002F   Decimal 0111 0047   o/   o/

O/   Hex 004F 002F   Decimal 0079 0111   O/   O/

The "slashed O" (upper & lower case?) you're seeing would appear to be the ø at Hex 00F8/Decimal 0248 and Ø at Hex 00D8/Decimal 0216, both of which are within the "extended ANSI range" ≤256 and the characters should be in the Windows character code pages for most Latin based fonts since WinXP, but may not be there for some "novelty fonts."

I don't have info on the innards of character maps for Macs.

The "slashed L" characters don't appear with a "full slash" in my favorite fonts, but possible "real chars" might be:

ł at Hex 0142 Decimal 0322
Ł at Hex 0141 Decimal 0321
ƚ at Hex 019A Decimal 0410
Ƚ at Hex 023D Decimal 0573

I would expect that even a Mac would have these chars in fonts, and would use the character form rather than an overstrike emulation of it in any program that actually perfomed a conversion, but ...

John

Subject: RE: Tech: display bug
From: GUEST,Jon
Date: 14 Feb 11 - 03:55 PM

Have you checked in tools/add ones? I suppose there could be some extension/ plugin that would do that.

Subject: RE: Tech: display bug
From: GUEST,Grishka
Date: 14 Feb 11 - 03:36 PM

Jack, very strange indeed. Does it occur in the preview page as well? And with other sites?

Obviously someone is trying to "help" you. Try deactivating your plugins, one after the other.

Subject: RE: Tech: display bug
From: Jack Campin
Date: 14 Feb 11 - 02:07 PM

ArtfulCodger's suggestion about excessively-smart fonting sounds plausible, but I can't find any preference in Firefox that might change it. Setting the default encoding doesn't do anything.

The problem doesn't occur in Safari.

Subject: RE: Tech: display bug
From: Mr Red
Date: 14 Feb 11 - 05:06 AM

In Word (which I don't now use) I think you could undo the symbol-effect by backspacing; still a drag, tho...

In Word and Excel (presume Access etc) "cntrl Z" will undo the last auto insert, click twice to undo the manual insert if there was an auto-insert.

"cntrl Z" is just an undo previous action - part of the "cntrl C" "cntrl V" "cntrl X" set and "cntrl Y" (repeat last action). Not tried backspace I would assume it deletes the character, but there you go.

Subject: RE: Tech: display bug
From: Artful Codger
Date: 14 Feb 11 - 04:07 AM

Another possibility: "smart" fonting examines adjacent text in order to determine the best display. In particular, it may recognize certain character combinations as digraphs that should be ligated, crowded together as a single letter (like ch or ll in some languages) or otherwise displayed differently (like using a different terminal form). Your display problem may be the result of your font/fonting agent being TOO smart. To prevent such interpretations, a Combining Grapheme Joiner (U+034F—really a dis-joiner) can be inserted between such characters. But don't expect anything like that to happen here! Are you using any unusual locale settings or fonts?

Subject: RE: Tech: display bug
From: JohnInKansas
Date: 14 Feb 11 - 02:20 AM

When Word autocorrect or autotype changes what you type to something it imagines is prettier, Ctl-Z "escapes" back one step, just like "edit-undo" and "erases" only the change if you do it before typing on, leaving what you typed in the original typed form.

The broken chars may be tied into an autocorrect/autoreplace function that's working in a browser, in which case there's probably a "backup" quick-key that may work - - maybe even Ctl-Z since it's a fairly universal "undo." Or you may need to change settings. Since others don't appear to see what you do, it's unlikely you're actually getting "wrong chars." You could check that by copying the char pairs you're seeing wrong and pasting them into Word or Notepad. If they show two separate chars there then they haven't been converted to a "combination glyph."

IE, and probably other browsers, do sometimes have a difficulty with "rendering" some fonts at small point sizes. This is due to there just not being enough pixels to draw the chars correctly with the screen resolution (75 or 96 dpi?) being too low for the character size. Officially the reported "break point" for this error is at about 6 pt type size at 96 dpi, which is pretty small (about .08 inch tall?); but it may vary with fonts and with monitors. You could try changing the scale at which you view the web page to see if that might be the cause, and/or change the text display size (small to larger?) in the browser.

If it's a screen resolution difficulty, a different font might help, since some forms are less "pixel loaded" than others.

John

Subject: RE: display bug
From: Jim Dixon
Date: 13 Feb 11 - 10:27 PM

If you don't like the way Word changes certain things, you can stop it. Go to Tools/AutoCorrect. You will see a substitution table. For example, the first item I see tells Word to change "(c)" to "©". If you don't want it to do that, just click on that line and then click delete.

You can also add lines to the table. You could, for example, make it change "dt" to "Digital Tradition Database".

Of course, all this only applies to stuff you type into a Word document, while you're typing it. It won't change any document retroactively. If you want to change a document retroactively, you can use AutoFormat on the Format menu.

Subject: RE: display bug
From: Artful Codger
Date: 13 Feb 11 - 09:27 PM

Looking at the page sources, I don't see anything unusual in the character representation. If there were something like an embedded backspace character between the two characters, that might explain it, but I wouldn't expect it to happen TWICE. In other words, I can see it creeping into the thread title, and thus getting replicated to every subject line, but when people type "o/" in their messages, I wouldn't expect such a thing to creep in again.

So my bet is that it's with your browser's display. For instance, you may have a setting (or add-on module) which tries to display accented characters emulated in ASCII the way they were intended to be rendered. Compare, for instance, the way you have to encode accented characters in ABC, or the way that h is used in Irish and Esperanto to emulate, respectively, lenition (dotted consonants) and circumflexed consonants/u-breve.

In any case, I doubt the Mudcat input is wrong or that the behavior you see has anything to do with the "obscure Tech: threads" issues (by which I think Foolstroupe means the threads on how to enter non-ASCII characters safely). As for whether Mudcat should apply an explicit encoding to its messages, that's quite germane to those issues, but probably irrelevant to yours.

Subject: RE: display bug
From: The Fooles Troupe
Date: 13 Feb 11 - 08:04 PM

That's being discussed in some obscure Tech: threads with weird names now, Jack.

Subject: RE: display bug
From: Jack Campin
Date: 13 Feb 11 - 06:34 PM

Oops. Copied the wrong link my second post. This is the right one:

thread.cfm?threadid=135621&messages=8

Can't be anything to do with my keyboard, as this isn't to do with what I type. I can see the HTML source - typed by other people - and it doesn't have Norwegian o's or Polish l's in it.

As far as I can see, Mudcat pages don't declare a character encoding. Perhaps they ought to.

Subject: RE: display bug
From: The Fooles Troupe
Date: 13 Feb 11 - 05:49 PM

Posts have been vanishing again - so here again...

On PC - Windows and Linux - if you have the keyboard set as something incompatible, it may give weird encodings - don;t know hwere to look for that on a Mac

Subject: RE: display bug
From: GUEST, topsie
Date: 13 Feb 11 - 08:29 AM

I can't imagine ever wanting to type :-), but (c) is fairly common in alphabetical lists.

Subject: RE: display bug
From: Bonnie Shaljean
Date: 13 Feb 11 - 08:25 AM

In Word (which I don't now use) I think you could undo the symbol-effect by backspacing; still a drag, tho...

Subject: RE: display bug
From: Bonnie Shaljean
Date: 13 Feb 11 - 08:24 AM

I now have Leopard (which works fine) but had Tiger until fairly recently (the final version, 10.4.11 I think) and used Firefox regularly, but never had that happen...? Assume this problem is something that has just cropped up?

The only thing I can think of is if the new FF has some sort of "auto-correct" option like Word & Open Office do, where it makes a symbol of two or three adjacent keystrokes unless to tell it not to. That happens on certain websites with those smiley faces, where :-) automatically turns yellow and gurns at you. (I can just imagine your opinion of those, Jack!)

Subject: RE: display bug
From: GUEST, topsie
Date: 13 Feb 11 - 07:26 AM

I sometimes get a copyright symbol when I try to type a c in parentheses. The solution is ti right-click on the unwanted character, and you get a little box with an opption to "stop changing ... "

Subject: RE: display bug
From: Leadfingers
Date: 13 Feb 11 - 07:14 AM

I'm on Windows XP and Internet Express and have normal display Jack

Subject: RE: display bug
From: Jack Campin
Date: 13 Feb 11 - 06:35 AM

Another place it happens: lowercase l followed by a slash gets turned into a Polish slashed-l, as in this thread:

thread.cfm?threadid=20634&messages=66

I assume it's a character encoding bug, but is the problem with what Mudcat is sending or with what Firefox does with it?

Subject: display bug
From: Jack Campin
Date: 13 Feb 11 - 06:31 AM

I'm using Firefox on MacOS Tiger.

When it tries to display a lower-case letter o followed by a slash, it shows me a Norwegian o-with-a-slash-through-it instead. As in the title for a current thread, "Who/what is the Great Speckled Bird?" - the title bar gets it right, text in the body of the page gets Norwegified.

thread.cfm?threadid=20634&messages=66

This happens with the character encoding set to either Western (ISO 8859-1) or Unicode (UTF-8).

How do I fix this?