Tech: CopyUnicode: Create any char

Subject: Tech: CopyUnicode: Create any char
From: GUEST,Grishka
Date: 11 Feb 11 - 12:21 PM

---

CopyUnicode is a utility which helps you enter individual characters in a portable way. To use it, you start the program, scroll to the character you want, click it, then paste to the Mudcat message entry area. Full directions for downloading and running the utility are given here in the second message.

I will attempt to keep the script and information up-to-date. I may also remove test posts and eliminate or correct faulty advice in situ. To test your messages, please use the Preview option; submit test messages only if you need someone tech-savvy to check your results (since what you see may not show the same way to other users.)

-Artful Codger-
[Source script updated 12 Feb 2011.]

---

On the Permathread about HTML character codes, I boastfully offered the following tool, assuming it would be rejected. However, Artful Codger asked me to post it in a separate thread, so here it is, FWIW. I extracted the following Java source code from a larger utility, which has been developped by a group of friends including myself. We donate this to the public domain, anyone is free to improve it.

Those of us who can profit most of this tool are likely not to have a Java compiler at hand. Therefore I suggest that Mudcat offer a compiled version (CopyUnicode.jar) for download (the same applies to Artful's htmlesc.java tool), which then can be used as an executable file on all up-to-date computers (Windows, Mac, Linux, ...).

When you invoke the program, usually by double-clicking on an icon etc., you will see a scrollable table displaying all the two-byte Unicode characters. If you click on one of these representations, the character resp. its HTML representation (looking like &...;) will be copied to the clipboard. If you then click into the Mudcat entry box and press Ctrl-V, the result will be inserted. You can deselect the HTML conversion and thus obtain the actual character, to inserted in other entry boxes or word processor systems etc.

I tried to keep the code short, so that experts can easily read it. Some bells and whistles may be found lacking. If mnemonic entities are desired, you can easily add this feature, as seen in htmlesc.

Actually I cannot believe that we are the first to invent this tool. If you know another implementation, feel free to post a link to it here; may the best win. Tell us about your experiences.

import java.awt.*;
import java.awt.datatransfer.StringSelection;
import java.awt.event.*;
import java.util.Arrays;
import java.util.Comparator;
import javax.swing.*;
import javax.swing.table.AbstractTableModel;

/**
 * @author  Grishka and friends
 * Donated to the public domain, feel free to improve
 */
public class ClipUnicode extends JFrame
{
  JCheckBox mHTMLCheckBox = new JCheckBox ();
  JComboBox mFontComboBox = new JComboBox ();
  JScrollPane mScrollPane = new JScrollPane ();
  JTable mCharsTable;

  boolean mIsFilling = false;

  String [] mFontNames;
  int [] mFontQualities;
  Integer [] mFontSortedIndices;
 
  private ClipUnicode ()
  {
    setTitle ("Copy a Unicode character to the clipboard");
    setDefaultCloseOperation (javax.swing.WindowConstants.EXIT_ON_CLOSE);
    getContentPane ().setLayout (new GridBagLayout ());

    JLabel lLabel = new JLabel ();
    lLabel.setText ("Instruction: Click on a character, click to the " +
            "destination, press Ctrl-V");
    lLabel.setToolTipText ("Donated to the public domain by Grishka and " +
        "Friends. Source code available at www.mudcat.org");
    GridBagConstraints lConstraints = new GridBagConstraints (0, 0,
        GridBagConstraints.REMAINDER, 1,
        1., 1., GridBagConstraints.CENTER, GridBagConstraints.NONE,
        new Insets (0, 0, 3, 0), 0, 0);
    getContentPane ().add (lLabel, lConstraints);

    // checkbox ======================
    mHTMLCheckBox.setText ("Copy the HTML code (otherwise the pure character)");
    mHTMLCheckBox.setToolTipText ("If selected, \"&#x...;\" will be copied, " +
        "otherwise a single character, no font information.");
    lConstraints = new GridBagConstraints (1, 1, 1, 1,
        0., 0., GridBagConstraints.NORTH, GridBagConstraints.NONE,
        new Insets (0, 0, 0, 0), 0, 0);
    getContentPane ().add (mHTMLCheckBox, lConstraints);
    mHTMLCheckBox.setSelected (true);

    // combobox to list the available fonts ======================
    mFontComboBox.addActionListener (new ActionListener ()
    {
      public void actionPerformed (ActionEvent evt)
      {
        if (!mIsFilling)
          SetView (mFontComboBox.getSelectedIndex ());
      }
    });

    lConstraints = new GridBagConstraints (2, 1, 1, 1,
        1., 1., GridBagConstraints.CENTER, GridBagConstraints.HORIZONTAL,
        new Insets (3, 5, 3, 5), 0, 0);
    getContentPane ().add (mFontComboBox, lConstraints);

    // fill it
    SortTheFonts ();
    mIsFilling = true;
    for (Integer lFont : mFontSortedIndices)
    {
      if (mFontQualities [lFont] < 10 && mFontComboBox.getItemCount () != 0)
        break;
      mFontComboBox.addItem (mFontNames [lFont]
          + " (" + mFontQualities [lFont] + ")");
    }
    mFontComboBox.setSelectedIndex (0);
    mFontComboBox.setToolTipText ("Font to use in this display, " +
        "good ones first in the list");
    mIsFilling = false;

    // scroll pane for the character table ======================
    mScrollPane.setPreferredSize (new Dimension (650, 600));
    lConstraints = new GridBagConstraints (0, 2, GridBagConstraints.REMAINDER, 1,
        1., 1., GridBagConstraints.CENTER, GridBagConstraints.NONE,
        new Insets (0, 0, 0, 0), 0, 0);
    getContentPane ().add (mScrollPane, lConstraints);

    // table of characters ======================
    CharTableModel lModel = new CharTableModel ();
    mCharsTable = new javax.swing.JTable (lModel)
    {
      @Override
      public String getToolTipText (MouseEvent aEvt)
      {
        Point lPt = aEvt.getPoint();
        int lRow = rowAtPoint (lPt);
        int lCol = convertColumnIndexToModel (columnAtPoint(lPt));
        if (lCol == 0)
          return "";
        int aCode = lRow * 16 + lCol - 1;
        String lRc = "x" + Integer.toHexString (aCode).toUpperCase () + " ("
                 + Integer.toString (aCode) + ")";
        Character.UnicodeBlock lUB = Character.UnicodeBlock.of (aCode);
        if (lUB != null)
          lRc += ", " + lUB.toString ();
        int lVal = Character.getNumericValue (aCode);
        if (lVal >= 0)
          lRc += ", numeric value " + Integer.toString (lVal);
        return lRc;
      }
    };
    mCharsTable.setRowHeight (25);
    mCharsTable.setCellSelectionEnabled (true);
    SetView (0);

    // action! ======================
    mCharsTable.addMouseListener (new MouseListener () {
      public void mousePressed (MouseEvent aEvt)
      {
        String lToClip = "";
        int lLow = mCharsTable.getSelectedColumn () - 1;
        if (lLow >= 0)
        {
          int lCodepoint = 16 * mCharsTable.getSelectedRow () + lLow;
          lToClip = (mHTMLCheckBox.isSelected ())?
              "&#x" + Integer.toHexString (lCodepoint).toUpperCase () + ";"
            : String.valueOf ((char) lCodepoint);
        }
        else
          mCharsTable.clearSelection ();
        StringSelection lSel = new StringSelection (lToClip);
        Toolkit.getDefaultToolkit ().getSystemClipboard ()
            .setContents (lSel, lSel);  
      }

      public void mouseClicked (MouseEvent aEvt) {}
      public void mouseReleased (MouseEvent aEvt) {}
      public void mouseEntered (MouseEvent aEvt) {}
      public void mouseExited (MouseEvent aEvt) {}
    });

    pack ();
  }

  private void SortTheFonts () // good ones first
  {
    GraphicsEnvironment lEnv = GraphicsEnvironment.getLocalGraphicsEnvironment ();
    mFontNames = lEnv.getAvailableFontFamilyNames ();
    mFontQualities = new int [mFontNames.length];
    mFontSortedIndices = new Integer [mFontNames.length];
    String lQualityTest = "\u00e1\u00e4\u00e7\u00df\u010d\u0142\u0119\u00a9" +
          "\u044f\u03c9\u00e5\u00bf\u00a3\u01dd\u20ac\u05d0\u0621\u266d" +
          "\u0e01\u1100\u2708\u3041\u4e00\uac00\u27f6";
    for (int lIdx = 0; lIdx < mFontNames.length; lIdx++)
    {
      mFontSortedIndices [lIdx] = lIdx;
      Font lFontSample = new Font (mFontNames [lIdx], 0, 10);
      mFontQualities [lIdx] = 2 * lFontSample.canDisplayUpTo (lQualityTest);
      if (mFontQualities [lIdx] < 0) // can display all of lQualityTest
        mFontQualities [lIdx] = 2 * lQualityTest.length ();
      if (mFontNames [lIdx].toLowerCase ().contains ("unicode"))
        mFontQualities [lIdx]++;
    }
    Arrays.sort (mFontSortedIndices, new Comparator ()
    {
      public int compare (Object aObj1, Object aObj2)
      {
        int lDiff = mFontQualities [(Integer) aObj2] - mFontQualities [(Integer) aObj1];
        if (lDiff != 0)
          return lDiff;
        return mFontNames [(Integer) aObj1].compareTo (mFontNames [(Integer) aObj2]);
      }
    });
  }

  private void SetView (int aFontIdx)
  {
    mCharsTable.setFont (new Font (mFontNames [mFontSortedIndices [aFontIdx]], 0, 20));
    mScrollPane.setViewportView (mCharsTable);
  }

  public static void main (final String args [])
  {
    EventQueue.invokeLater (new Runnable () {
      public void run ()
      {
        new ClipUnicode ().setVisible (true);
      }
    });
  }

  class CharTableModel extends AbstractTableModel
  {
    Component mComponent = null;
    Color mBgRowHeaders = new Color (240, 240, 240);
    Color mBgUppercase = new Color (230, 230, 255);
    Color mBgLowercase = new Color (255, 230, 230);
    Color mBgOtherLetter = new Color (255, 240, 215);
    Color mBgDigit = new Color (230, 255, 230);
    Color mBgElse = Color.white;
    Color mCurrBg = null;

    public int getRowCount() { return 256 * 16; }
    public int getColumnCount () { return 17; }
    @Override
    public String getColumnName (int aCol)
    {
      return (aCol == 0)? "" : Integer.toHexString (aCol - 1);
    }
    public Object getValueAt (int aRow, int aCol)
    {
      char lChar = (char) (aRow * 16 + aCol - 1);

      if (mComponent == null)
        mComponent = mCharsTable.getCellRenderer (0, 0)
            .getTableCellRendererComponent (mCharsTable, 0,
             rootPaneCheckingEnabled, rootPaneCheckingEnabled, 0, 0);
      if (aCol == 0)
      {
        mComponent.setBackground (mCurrBg = mBgRowHeaders);
        return Integer.toHexString (aRow);
      }

      // set the background color according to the type of character
      Color lColor;
      if (Character.isUpperCase (lChar))
        lColor = mBgUppercase;
      else if (Character.isLowerCase (lChar))
        lColor = mBgLowercase;
      else if (Character.isLetter (lChar))
        lColor = mBgOtherLetter;
      else if (Character.isDigit (lChar))
        lColor = mBgDigit;
      else
        lColor = mBgElse;
      if (lColor != mCurrBg)
        mComponent.setBackground (mCurrBg = lColor);

      return lChar;
    }
  }
}

Subject: RE: Tech: CopyUnicode: Create any char
From: Artful Codger
Date: 11 Feb 11 - 06:32 PM

CopyUnicode Guide

For info on which characters you should be encoding, see the guide Entering special characters. Basically, if it's not in the following list, encode it:

A-Z a-z 0-9 ! " # $ % ' ( ) * + , - . / : ; = ? @ [ ] \ ^ _ ` { } | ~ space tab newline

Be particularly careful about quote, apostrophe, dash and elipses characters in text you copy/paste from other sources.

How to get CopyUnicode
Download the Java JAR from Jon Freeman's website. It is available in both 32-bit and 64-bit versions. The 32-bit version should work with any reasonably current system; the 64-bit version is more efficient, if your system can run it.

     64-bit JAR (needs Java version 1.6.0 or later)
     32-bit JAR

---

Alternative download locations:
Joe-web (Joe Offer): 64-bit, 32-bit
SkyDrive (Artful Codger): 64-bit, 32-bit

---

You can download the JAR directly to your desktop, or to any convenient location where you typically store programs. It can be run from a shortcut/alias/softlink, so you only need one copy on your system. If you need to run the JAR from the command line, however, you'll need to put it in a directory referenced by your JAVAPATH environment setting. The command-line command to run it is:
     java -jar CopyUnicode.jar

How to use CopyUnicode
Start the program by double-clicking on the JAR file (or shortcut/alias). A dialog appears, presenting a scrollable grid of characters (all the Unicode 16-bit characters). Ensure that the "Copy the HTML code" checkbox is checked. Browse to the character you want and click on it; an HTML sequence will be placed on your clipboard. Go to the Mudcat message entry area, click on it, then paste (Control-V on Windows, Command-V on Mac). In the entry area you should see a sequence like é—you should not see the character you selected, but rather something starting with &#x and ending with semicolon, with a few hexadecimal digits (which include A-F) in between. If you see the actual character, delete it, check the checkbox and try again.

To verify that what you've entered is what you want, use the Preview option below the message entry area: check it, then click Submit. In the upper part of the preview page, you should see the characters you want; in the lower part (the message entry area) you should still see the ampersand sequence instead.

As Grishka says, this tool can also be used to enter characters in other web pages, in word processors and in other programs (like spreadsheets) which can accept Unicode characters directly. In this case, just uncheck the "Copy the HTML code" checkbox, and you'll get the actual Unicode character instead.

Both Mac and Windows have OS utilities for browsing the character maps and copying characters to the clipboard. (On Mac, it's the Character Palette.) They also report the Unicode value, from which one can form a character reference by enclosing the value in "&#x" and ";". But they won't copy the character automatically for you in an HTML-encoded form, which is the only portable way to enter your text in Mudcat.

Subject: RE: Tech: CopyUnicode: Create any char
From: Bill D
Date: 11 Feb 11 - 06:57 PM

It's not cross-platform and java, but for Windows, this program works quite well and is customizable for each person.

This one also is very nice, though it requires you to know the code each time.

Subject: RE: Tech: CopyUnicode: Create any char
From: GUEST,Jon
Date: 11 Feb 11 - 09:37 PM

64bit version compiled using java-1.6.0-Sun here if anyone wants to grab it.

Subject: RE: Tech: CopyUnicode: Create any char
From: GUEST,Jon
Date: 11 Feb 11 - 09:42 PM

Ħĕĺļö

Probably should have added, to run it:

java -jar CopyUnicode.jar

Subject: RE: Tech: CopyUnicode: Create any char
From: GUEST,Jon
Date: 11 Feb 11 - 10:07 PM

32 bit jar file here

Subject: RE: Tech: CopyUnicode: Create any char
From: GUEST,Jon
Date: 11 Feb 11 - 10:22 PM

The program is the lines of java code Grishka gave.   I copied and pasted that into Netbeans which compiled it and turned out the jar (which is actually like a zip file btw - you can open it and view its contents although most will be binary files...) which you downloaded and double clicked...

Subject: RE: Tech: CopyUnicode: Create any char
From: Artful Codger
Date: 11 Feb 11 - 11:14 PM

BillD: Unchecking the checkbox (that is, pasting in the raw character) is unsafe, leading in many cases to the very display problems that the escape sequence resolves. It happened to work because those characters weren't in your default codepage, so the input system encoded them for you. But the characters you'd most often use the tool for probably are in your codepage, so you'd be pasting in non-portable values for them. For the full scoop, see my guide (link in the second message).

Subject: RE: Tech: CopyUnicode: Create any char
From: GUEST,Grishka
Date: 12 Feb 11 - 08:47 AM

---

Grishka fixed a bug and reposted the code here. Jon has incorporated the changes into the JAR files, so if you fetch them now, they'll have the fix. I've updated the script accordingly above, and deleted the content here for brevity.

-Artful Codger-

Subject: RE: Tech: CopyUnicode: Create any char
From: GUEST,Grishka
Date: 12 Feb 11 - 09:11 AM

The "HTML" checkbox is initially checked (selected) and focused; if you (accidentally or not) press the space bar before clicking elsewhere, it will be deselected.

How to make a "jar" on a Windows system without knowing Java, NetBeans, or Eclipse:

1. Download the JDK and install it without changing the preset target directory


2. Copy the above source code from the browser window to a plain ASCII file named ClipUnicode.java, placed in an empty folder


3. In the same folder, create another text file named something.bat containing the commands below


4. Double click it


5. A third file named ClipUnicode.jar will appear. move, copy or upload it where you want it, delete the other two files if no longer needed


6. Uninstall the JDK if no longer needed.

The contents of something.bat are as follows - important: you may have to change the two occurences of "jdk1.6.0_23" according to your version.

mkdir Classes
"%ProgramFiles%\java\jdk1.6.0_23\bin\javac" -d Classes ClipUnicode.java

echo Manifest-Version: 1.0> manifest.mf
echo Main-Class: ClipUnicode>> manifest.mf
echo Class-Path: >> manifest.mf

cd Classes
"%ProgramFiles%\java\jdk1.6.0_23\bin\jar" cmf ..\manifest.mf ..\ClipUnicode.jar *

cd ..
del Classes\*.class
rd Classes
del manifest.mf

Subject: RE: Tech: CopyUnicode: Create any char
From: GUEST,Grishka
Date: 12 Feb 11 - 09:24 AM

Note that if you run anything executable means that you implicitely trust it. It is easy to convince yourself that my code above does no harm, but if you download a "jar", you still have to trust that it actually represents my code.

Enjoy the tool.

Subject: RE: Tech: CopyUnicode: Create any char
From: Bill D
Date: 12 Feb 11 - 02:28 PM

?? so... do I need to get a slightly revised version of that .jar?

I am trying to avoid having to actually type the ampersand codes. Does the revised version allow this without unchecking the box?

(I am NOT a programmer and the guides tell me far more than I want to know. I find and use tools. I can drive almost any vehicle, but my mind does not wish to become a mechanic. This is why I loved the idea that I could download a compiled program and enter a character with a couple of clicks.) (I am in awe of those who write code...even when it takes a couple of tries)

Subject: RE: Tech: CopyUnicode: Create any char
From: Artful Codger
Date: 12 Feb 11 - 02:56 PM

To Grishka: Um, I think you meant "CopyUnicode" rather than "ClipUnicode".

---

No, his instructions are correct as given. It does mean that the JAR file produced will have a different name. Jon takes a different approach, which preserves the CopyUnicode name for the JAR. See Grishka's response below. --AC

---

IMPORTANT: Make sure the checkbox is checked when copying to Mudcat. If you don't, you'll see the raw character in the message entry area instead of a sequence beginning with ampersand. This means you're doing it wrong!

For instance, above, where Jon pasted the odd-looking "Hello", the characters were inserted raw. The input system did properly encode the first four characters automatically, but the o-umlaut was not encoded. Switch your view encoding for this page to, say, MacRoman (a common default for Mac users), and instead you'll see a lone circumflex accent. If you're encoding things properly, you should see all the text correctly no matter which view encoding is in effect.

Subject: RE: Tech: CopyUnicode: Create any char
From: Artful Codger
Date: 12 Feb 11 - 03:01 PM

Bill, yes and yes. Get the new version of the JAR, and then when the box is checked and you click/paste, what you'll see in the entry box should be something like "ý" rather than the character itself. If you preview your message, you'll see the desired character in the preview area. Better still, so will everyone else, when your message is posted.

Subject: RE: Tech: CopyUnicode: Create any char
From: GUEST,Grishka
Date: 12 Feb 11 - 03:25 PM

Bill, the answer is yes. If I understand Jon 11 Feb 11 - 11:09 PM correctly, he has uploaded a version corrected by himself here, for which we owe him a thankyou. If you feel like trusting him (which you did before), just download again; it should now do exactly what you want. Maybe one of the tooltip texts is not quite what it should be, because it also contains that &#x, but this cannot impair the functionality. I guess Jon will do a third shot, in which case we owe him another thankyou. And I owe a Sorry to all of you, which I herewith pay. (That was my second blunder yesterday at Mudcat. If we hadn't known before, we would have learned that anyone can have a bad day. Glad it's weekend now.)

BTW: A jar file is one-size-fits-all, Windows 32, 64, 128, Linux, Unix, Solarix, Asterix, MacOsX ... Nevertheless it may look and work slightly differently or even fail on some systems. Please report.

---

The tooltip change is now reflected in the script source shown in the first message. Jon has incorporated the change into the 64-bit JAR.

-Artful Codger-

Subject: RE: Tech: CopyUnicode: Create any char
From: GUEST,Jon
Date: 12 Feb 11 - 03:37 PM

OK. 64bit compilation has been updated for the toolips (using Grishka's second code version).

I'm not sure about the need for compiling a 32bit version now. I had thought there could be problems but maybe I'm just thinking of problems I've had with applet plugins and browsers?

Subject: RE: Tech: CopyUnicode: Create any char
From: GUEST,Grishka
Date: 12 Feb 11 - 03:53 PM

On refreshing, I see the other answers.

Thank you, Jon. If anyone finds any incompatibilities, let me know.

Artful Codger (12 Feb 11 - 02:56 PM), you can call the program as you wish. Ok, let us stick to CopyUnicode in this thread, but the main class is ("historically") called ClipUnicode in the above source code, so if you want to call the source file CopyUnicode.java, the above something.bat should read:

mkdir Classes
"%ProgramFiles%\java\jdk1.6.0_23\bin\javac" -d Classes CopyUnicode.java

echo Manifest-Version: 1.0> manifest.mf
echo Main-Class: ClipUnicode>> manifest.mf
echo Class-Path: >> manifest.mf

cd Classes
"%ProgramFiles%\java\jdk1.6.0_23\bin\jar" cmf ..\manifest.mf ..\CopyUnicode.jar *

cd ..
del Classes\*.class
rd Classes
del manifest.mf