 sj |
||
|
||
Tech: converting binary data to ASCII
|
|
|
Subject: Tech: converting binary data to ASCII From: TIA Date: 23 Oct 08 - 10:25 PM Hi all you folks that are smarter than me. I have an accelerometer up and running sending data to my RS-232 (COM1) - capturing data in Hyperterminal as a txt file. I want to collect at the maximum possible sampling rate, so according to the spec. sheet for the accelerometer, I need to run in Binary Output mode. Does anyone have a routine, or know of freeware, that can convert the binary to ASCII? It would be nice if it was user friendly, 'cause while I know vibration science and music, I am not very computer savvy. I really appreciate any and all tips. I do know exactly the format of the binary (i.e. what bit corresponds to what piece of info, etc.) Thanks, Tim BTW, if you are wondering what the heck this has to do with music, the application is to detect (if any) sympathetic vibrations between buried landmines and a tuba, and to see if they are different from the vibes between a landmine and e.g. a rock, or a bit of scrap metal. No kidding. My geeky daughter is either genius or insane, or both. |
|
Subject: RE: Tech: converting binary data to ASCII From: Gulliver Date: 24 Oct 08 - 12:04 AM I spent a lot of time developing a machine to sort out this very issue, but then the wheels fell off... |
|
Subject: RE: Tech: converting binary data to ASCII From: GUEST,Peace Date: 24 Oct 08 - 01:15 AM www.theskull.com/javascript/ascii-binary.html Google that and see if it helps. If not, try to contact the folks who made that site. |
|
Subject: RE: Tech: converting binary data to ASCII From: GUEST,Peace Date: 24 Oct 08 - 01:16 AM And please don't tell Joe Offer I had ANYthing to do with this thread. Your friend, Dave. |
|
Subject: RE: Tech: converting binary data to ASCII From: Joe Offer Date: 24 Oct 08 - 01:34 AM Peace??? Giving TECHNICAL advice? Aaaaaaaaarrrrrgh!!!! Sincerely, -Joe Offer- |
|
Subject: RE: Tech: converting binary data to ASCII From: pavane Date: 24 Oct 08 - 02:01 AM Not sure what you actually MEAN. ASCII is equivalent to letters, each of which occupies 8 bits (In the old MSDOS or UNIX version) It appears that your data consists of binary numbers and other single-bit indicators. So what I presume you need is something to convert your binary numbers to decimal representation and / or your bits to text descriptions? This would need a program to be written for that specific task. Quite straightforward, as long as the orgrammer knows the exact form of the data. |
|
Subject: RE: Tech: converting binary data to ASCII From: pavane Date: 24 Oct 08 - 02:02 AM PROGRAMMER, not orgrammer!!!! |
|
Subject: RE: Tech: converting binary data to ASCII From: GUEST,.gargoyle Date: 24 Oct 08 - 03:59 AM It was a standard in the Old, Old, Norton Utilities.
You would have two columns.
One side would be hex and on the other side ANSI
It permitted you to patch broken strings of data. No doubt it is still around.
Sincerely,
"sympathetic vibrations between buried landmines and a tuba ... could this help explain Canadian troops carrying bagpipes? |
|
Subject: RE: Tech: converting binary data to ASCII From: Mitch the Bass Date: 24 Oct 08 - 04:43 AM There's a whole list of serial port utilities at http://rs232.qarchive.org/ some of which look like they may meet your requirement. Mitch |
|
Subject: RE: Tech: converting binary data to ASCII From: The Fooles Troupe Date: 24 Oct 08 - 05:26 AM Actually, I seem to remember that there was more than one ASCII standard... (old fart)... |
|
Subject: RE: Tech: converting binary data to ASCII From: JohnInKansas Date: 24 Oct 08 - 08:06 AM You are collecting numerical information as binary numbers. You DO NOT want to convert to ASCII. You want to convert binary numbers to decimal numbers. (I think) An example of conversion of binary to ASCII can be found at Binary to ASCII. (Found accidentally from the Worth Scary Signs 4 site, where he suggests pasting the roadsign text" 0110001001110010011010 0101100100011001110110 0101001000000110001101 1011000110111101110011 0110010101100100001000 0001100001011010000110 01010110000101100100 shown on the 25th sign in this collection at the above site to get the ASCII translation. This sign is nearer the bottom of the page than the top.) ASCII is a system that assigns numbers to letters, so that each letter is represented by a specific number. The ASCII system is based on 8-bit numbers, and can thus represent only 256 different "characters." This was sufficient for a basic alphabet and a few "control codes" but was replaced decades ago by the ANSI system that allowed an "extra bits" to get more characters. Old timers like Foolsetroupe (and maybe a few others who've studied history) will recall that in DOS and ancient Windows times, one had to add the "ANSI SYS" line to "AUTOEXEC.BAT" to boot your computer so it could read the extended text characters (largely graphical symbols) that were then beginning to be used some. (And back then you might have to add "extended memory" and also put HIMEM.SYS in the bootup files as well, to get past the 64 KB max RAM that "old DOS" (and earliest Windows) could read.) Most people now ignore the difference between ASCII and ANSI, so when they say ASCII they probably actually mean ANSI. The only difference is "how many characters" you're allowed to use in your "alphabet." Note that IBM (and a few others) did not (back then) use ASCII character codes, but used instead a system called "ebcdyk" or something like that, which completely scrambled text when sent between IBM and PC computers, unless a "translator" was used. The first thing you need to know is how many bits are in each number. Common instruments may use 8 or 16 bits, although occasionally you'll fine "n-1" coding that give you only 7 or 15 "useful bits." The second thing you'll need to know is whether the device outputs "Big-endian" or "Little-endian" data. Some devices write the "least significant bit" first and some write "most significant bit first" which means the numbers, as printed from raw data, may read right-to-left or left-to-right. (Internally, PCs and Macs are (or were in the original versions) "opposites" for this division, but you never see "raw" binary, so it doesn't matter much. The original Motorola MC6800 processors used in Macs caused the "flip" but Mac has now gone mostly(?) to Intel processors which may or may not have made them "reverse" the ends.) Although the conversion from binary to decimal numbers is simple, there is no standard simple program that I've seen for automatically translating binary numbers to decimal numbers; but you can "learn to read" binary fairly easily, or you can use a chart to translate them. You want to convert:
In binary, note that as the number is incremented to the next larger one, the least significant bit is flipped from 0 to 1 or from 1 to 0. If the "flip" is from 0 to 1 the next bit to the left doesn't change. If the "flip" is from 1 to zero, the next bit to the left is also "flipped," If you get rid of the "ASCII fixation" and put "Binary to Decimal" into a Google search, you should come up with something like This Google Result which may find a "calculator" you can use for your conversion. Given that most instruments are very much faster than "sonic speed," if your accelerometer is capable of giving you direct decimal values in its output, it is questionable whether your test benefits from outputting binary, given the extra work you'll do in conversion. Many instrument devices (or their interface "amps") include BCD ("Binary Coded Decimal" if you need a search term) converters, and "doing it in hardware" is generally faster than "computing it" and probably is sufficient for the test described. You're resident genius needs to look at what's required to get useful data for the test being conducted, and not be hung up on "what's the most precise that's possible," perhaps. An (almost trivial) advantage to using binary output is that if the binary numbers are lined up in a column (especially using a monospace font like Courier) a line through the "most significant bits" (i.e. the left-most "1"s) is a simple and fairly effective, if crude, "graph" of the outputs, so with a little practice you can "read" the outputs quickly to see the changes in the data. It takes pretty long printout to get useful "graphs," if you have a high sampling rate, but manual (or programmed) conversion of "spot values" may be sufficient - if you choose to keep the binary output for this purpose. John |
|
Subject: RE: Tech: converting binary data to ASCII From: John J Date: 24 Oct 08 - 08:23 AM There's 10 types of people who understand binary: those that can and those that can't. .....I'll get me coat. JJ |
|
Subject: RE: Tech: converting binary data to ASCII From: pavane Date: 24 Oct 08 - 10:16 AM Pretty much what I was trying to say - you need a program to give you the DECIMAL representation of the BINARY number, not ASCII at all. |
|
Subject: RE: Tech: converting binary data to ASCII From: JohnInKansas Date: 24 Oct 08 - 10:44 AM If the "places" in the binary are number are numbered (in decimal) from least significant to most significant, then the "bit-value" in each place contributes (bit-value)*2n-1, where n is of course the "place-value" (i.e. the column) for the bit. In the first (least significant) place, n = 1, n-1 = 0, 20 = 1, so if the bit-value is zero, 0 * 1 = 0, and if it's 1, 1 * 1 = 1. For the next bit, n = 2, n-1 = 1, 21 = 2, so if the bit-value is zero, again 0 * 2 = 0, and if it's 1, 1 * 2 = 2. Since 0 x (anything) is zero, you only have to look at the places with "1"s in them.
If you simply strike out the "places" where there are zeros, and add the above values for all the "places" where there are ones, the total is the value of the binary. Thus the binary 10101010 is 128+32+8+2 = 170 in decimal notation. And the binary 01010101 is 64+16+4+1 = 85 in decimal notation. And binary 01101101 is 64+32+8+4+1 = 109 in decimal notation. Really simple, once you get the hang of it. Someone will now notice that the binary 11111111 has decimal value 255, and we said that an 8-bit binary can have 256 different vales. Before someone pops a fuse, note that one of the 256 values is ZERO. With 8 bits, one has 256 "values" and with 256 "values" one can only count from zero to 255. Unless, as has been done by some of the computer wizards, you "shift" the results to make 00000000=256, (an implied "flip of the next left bit" to 1 - in the column with column-value 256, even if there is no such column, which when added to 00000000 gives decimal 256) so that you can count from 1 to 256, but can't sell a zero on the open market. It has been done both ways, and people have argued about which is right since about day 1 (or maybe it was day 0). John |
|
Subject: RE: Tech: converting binary data to ASCII From: TIA Date: 24 Oct 08 - 03:46 PM Wow, I've got some reading to do. Thanks. Actually, i have two problems. Yes, I do need to get from binary to decimal ultimately. But what I can capture in Hyperterminal (with my limited skills) is strings of funny characters (little smileys etc.) I presume this is because the binary is being represented in the captured txt file as ascii characters, not strings of 1's and 0's. So, I need to get the true binary, then turn it into decimal in an ASCII text file that I can open in Excel. I'll try to get me some education on this now. Thanks everyone for the links and advice. |
|
Subject: RE: Tech: converting binary data to ASCII From: jeffp Date: 24 Oct 08 - 03:53 PM John -- the "ebcdyk" was EBCDIC -- Extended Binary Coded Decimal Interchange Code. It was used on IBM mainframes, because it worked well with punched cards. |
|
Subject: RE: Tech: converting binary data to ASCII From: TIA Date: 24 Oct 08 - 03:57 PM John - your post is very helpful. The first thing I checked is do we really need to output in binary mode, and the answer is unfortunately - yes. Turns out that forward modelling suggests a resonant frequency somewhere between 100 and 300 Hz. So, to avoid aliasing, we need to sample at double the highest frequency of interest. That means our Nyquist frequency is 600 Hz minimum. The gizmo says it samples at 595 Hz at the highest baud rate and in "binary output mode". So, I'm stuck having to do it the hard way (if I can). |
|
Subject: RE: Tech: converting binary data to ASCII From: MartinRyan Date: 24 Oct 08 - 04:07 PM Forgive my ignorance but - what's an "acclerometer" in this context? Also - have you successfully captured ASCII output? I wouldn't worry about the Nyquist/aliasing problem until I had a clear idea of the general shape of the data anyway. Regards p.s I agree with JinK's analysis, BTW. |
|
Subject: RE: Tech: converting binary data to ASCII From: Amos Date: 24 Oct 08 - 05:25 PM THe accerometer is just what it sounds like--a device that measures changes in acceleration and sends out the measurements in a stream of digital bytes. A |
|
Subject: RE: Tech: converting binary data to ASCII From: MartinRyan Date: 24 Oct 08 - 05:47 PM OK - I'm with you now. My mind was heading towards particle ACCELERATORS, I'm afraid! Age is becoming a form of intellectual gravity... Regards |
|
Subject: RE: Tech: converting binary data to ASCII From: Bernard Date: 24 Oct 08 - 06:28 PM If you do successfuly capture ASCII output, keep it in a suitably ventilated box... they're very rare, you know. JJ, can I borrow your coat? |
|
Subject: RE: Tech: converting binary data to ASCII From: SPB-Cooperator Date: 24 Oct 08 - 06:38 PM I rather not remember... takes me back to my college days and the endless exercises - btw don't forget the parity bit. |
|
Subject: RE: Tech: converting binary data to ASCII From: Bernard Date: 24 Oct 08 - 07:28 PM Isn't she a woman who won't stop talking?! And a serial interface is a spoon... cereal into face... aaawh, you hadta be there...! |
|
Subject: RE: Tech: converting binary data to ASCII From: JohnInKansas Date: 24 Oct 08 - 08:12 PM TIA So you're using a strain gage accelerometer (or a piezo with a big diaphragm)? (That's not really a serious question, as it wouldn't affect anything I can say that would be helpful.) Obviously you have the number one problem of nearly all high-tech experimentation: "You need a bigger budget!" SPB (if I may use your first name without seeming overly friendly?) Parity bits have pretty much been discarded in more recent computers. The one that's most concern now is proper handling of the "dirty bit." Some say those are the ones that make all the other extra bits stick to the bottom of the bit bucket so that it overflows and gets bits all over the place. John |
|
Subject: RE: Tech: converting binary data to ASCII From: Howard Kaplan Date: 24 Oct 08 - 11:13 PM If the resonant frequency is above 100 Hz, then why are you using an accelerometer rather than a microphone? A microphone feeding a sound card can easily be captured as a .wav file, which immediately gives you the option of seeing the waveform as a graph using something like Audacity or Goldwave. For analysis in Excel, you would again need to find a program to translate the .wav file's binary words to another format, as discussed above, bypassing whatever words are file header rather than data. (In fact, Excel will store the words' values internally as binary numbers, but as a different kind of binary numbers -- floating point -- than the kind of binary numbers in the .wav file.) One approach would be to use an Excel VBA program to read the file words and populate the spreadsheet with their values. Although such a program would not be very large, it would not be a good starting project for someone who's never used VBA and who's new to the idea of binary-coded numbers. However, outsourcing the writing of this short program might be a reasonable strategy, if there's nothing suitable already out on the 'net. |
|
Subject: RE: Tech: converting binary data to ASCII From: George Seto - af221@chebucto.ns.ca Date: 25 Oct 08 - 09:46 AM Try checking this out (Not sure if it works in real-time) Binary to ASCII Converter |
|
Subject: RE: Tech: converting binary data to ASCII From: Artful Codger Date: 25 Oct 08 - 07:17 PM The simplest way would be with the UNIX utility od (octal dump), which converts from binary to octal/hexadecimal/ASCII. It can also function as a filter program, so that, on UNIX systems, you can convert data streams in real time. od is standard on UNIX, and can be obtained as freeware for DOS/Windows. However, DOS and Windows don't handle command pipes properly, requiring temporary files that are only processed after the input has stopped. That makes them generally unsuitable for real-time streaming. Not a problem if you're transferring later, after accumulating the data. But it sounds as if you're specifically dealing with real-time transfer. |
|
Subject: RE: Tech: converting binary data to ASCII From: TIA Date: 26 Oct 08 - 12:43 PM Good questions all - The accelerometer is actually a tiny piezo one to avoid adding mass the the item being measured (and thereby possibly altering the resonant frequency). There is tremendous signal loss in air relative to soil, so while the transmitter (Tuba) must be air-coupled, we will get much stronger singals by measuring vibration directly on the earth rather than trying to get the much weaker vibrations in air. Also, by using an acceleremoter, we avoid ambient noise. I'm afraid I do need to worry about the Nyqusit frequency right off the bat, because it affects the "shape" (I presume you mean the spectral shape?) of the sound. If I look at the shape first, I am already looking at possibly alias-affected spectra. The data coming in the RS-232 in raw mode is strings of 1's and 0's (duh). I do have a spec sheet that lists what each bit represents - i.e. the exact format - but I am no programmer, so I am looking for something that, if I record the strings, I can crank them through to get back decimal numbers, and store them as ASCII text, which I do know how to import to Excel. I am probably describing the computer part poorly, becasue my knowledge lies in spectral analysis of vibrations, not computer stuff. So, once I have decimals numbers in Excel, I'm good. Until then, I have data that I don't know what to do with. Arful Codger - I am actually interested in post-processing, not real-time conversion. I want the fastest sampling rate possible, and the real-time conversion (which can be done onboard the accelerometer) slows it down. It sounds like you have a lead on exactly what I am after, but I'm stuck in Windows! Do you know of anything that does exactly what you describe above, but for DOS or Windws? Again, I do not want to convert on-the-fly. I am happy to stream binary to some type of file through Hyperterminal, and convert later. I really appreciate all the good tips, and am trying to ejookate myself up to your levels. |
|
Subject: RE: Tech: converting binary data to ASCII From: DMcG Date: 26 Oct 08 - 12:56 PM True story: you have to be careful when projects like this start taking all your time. My eldest son's first three words were mama, dada and oscilloscope. |
|
Subject: RE: Tech: converting binary data to ASCII From: Artful Codger Date: 26 Oct 08 - 05:31 PM As I mentioned, you can obtain od for DOS/Windows as freeware. And as long as you're able to capture the streamed data from Hyperterminal to a file, the piping limitation I mentioned is a non-issue for post-processing. However, od only converts the raw bytes to character equivalents. It sounds as if you're dealing with formatted, record-oriented binary output, and to convert that you'd need a customized script. Such a thing could be hacked together fairly easily in a language like Perl or even (gag me) Visual Basic; Perl was specifically designed to handle conversion tasks like this, though it has evolved into much more. But as you're not a programmer, learning the language and writing the script might be heavy going. Another conversion consideration: If you intend to import into Excel, you must write the converted data into a proper database import format--a raw data file isn't sufficient. Fortunately, some formats, like CSV, are character-based and pretty simple; essentially, you'd just have to supply the table and field names, then the rows of data values with proper delimiters between, and indicate when the end of the table is reached with a blank line. On the other hand, I suspect that whatever you'd need Excel for could be done more easily and better by the converter script itself. |
|
Subject: RE: Tech: converting binary data to ASCII From: JohnInKansas Date: 26 Oct 08 - 06:25 PM After some poking about, which mostly revealed that I know nothing useful about what you need to do, I did stumble across what looks like a "preliminary draft" from someone doing what you need. The posted item is a Word document: Binary Data File Format for the MITes You can also view it in html at View as html It seems that you have getting the data into your computer pretty will accomplished. Your problem is getting the data into a readable form. This document describes how one lab apparently stores data like yours so that is accessible for analysis. Unfortuantely I don't recognize any of the programs mentioned, and it's likely that they're "home made" by this lab; but it seemed to me to give an outline of the kind of thing you want. Maybe. If it doesnt' suggest anything helpful, it probably won't take you too long to read and delete. John |
|
Subject: RE: Tech: converting binary data to ASCII From: George Seto - af221@chebucto.ns.ca Date: 26 Oct 08 - 07:37 PM TIA, Does your computer store the input from the accelerometer as a file?? Right now, I'm unclear if you are wanting to take the input from your RS232 port then convert it to ASCII then save it. OR if your computer is taking that input saving it as a binary file and you want to convert the binary file into your ASCII file. That might clarify things |
|
Subject: RE: Tech: converting binary data to ASCII From: TIA Date: 28 Oct 08 - 05:22 PM Artful Codger - I think you've got the picture exactly. I wish I was a programmer. That could solve it all. Thanks again all. John I will read the links you posted - I may be able to figure out something there - thanks. George - Hyperterminal is streaming the binary directly to a file. When I open this file in Notepad or anything else, it is just funny symbols. So, I am pretty sure it is a binary file. I need to convert it to something ASCII si I can import to Excel. |
|
Subject: RE: Tech: converting binary data to ASCII From: JohnInKansas Date: 28 Oct 08 - 06:51 PM TIA - Most likely what you need is a "file editor" and the most likely useful format you'll see in one is "hex" characters, with one pair of hexadecimal characters per 8-bit byte. (1 in decimal = 1 in hex, 2 decimal = 2 hex, ... 9 decimal = 9 hex, 10 decimal = A hex, 11 = B, 12 = C, 13 = D, 14 = E, 15 = F, 16 decimal = 10 hex, etc. For 8 bit bytes, each number will be a "pair" of "hex characters, like 01, ... 10, ... A4 ... AE etc. For 16 bit bytes you'll need "two pairs" to represent one "character.") Translating to ASCII is of little use, since many of the bits/bytes you want to look at won't have "unique glyphs" to represent them as text. The "funny characters" you see in Notepad are an ASCII representation of the bytes in the file. Excel should be able to accept pasted in Hexadecimal numbers (number pairs or pairs of pairs), or should open a file containing hex numbers if you can figure out how to get them in "one per cell," and you should be able to "format cells" as equivalent decimal numbers for a direct "translation". For Notepad to "display" the binary number 01010101 it would need to convert the bits to ASCII characters "0" and ASCII characters "1" - each a byte long, so it would need to "read" the decimal values 0030 0031 0030 0031 0030 0031 0030 0031 to display 01010101. The binary number 01010101 is a decimal 64+16+8+4+1 = 93, which the computer automatically "reads" as a hexadecimal 005D, which it translates into a "text glyph" in Notepad, which in this case - in most fonts - would be a square-right-bracket "]" or something of the sort. (Word - recent versions - can translate "hex" to "ASCII" or Unicode if you type the Unicode hex number for a character, and with the cursor immediately to the right of it, hit Alt-X. You should see the glyph that that number represents in a Unicode font. It's a toggle, so a "glyph" can be "read" by putting the cursor immediately adjacent on its right and hitting Alt-X to change it to the Unicode hexadecimal number that produces that glyph. Word, in versions where this works, uses 16 bit bytes, so you'll generally see four "hex digits.") (Now, I suspect, we're both confused?) John |
|
Subject: RE: Tech: converting binary data to ASCII From: GUEST,.gargoyle Date: 28 Oct 08 - 09:46 PM A refreshing thread to read.
Informative, Educational and Technical.
I never got beyound 8 bit...in my education...but I love to hear the big boy's talk.
Sincerely, |
|
Subject: RE: Tech: converting binary data to ASCII From: pavane Date: 29 Oct 08 - 02:32 AM Wishing to be a programmer is like wishing to be a musician - it doesn't get you very far. You need tuition and hard work! |
|
Subject: RE: Tech: converting binary data to ASCII From: Artful Codger Date: 29 Oct 08 - 03:30 PM Except that the data, in any character representation of the raw bytes, is unlikely to be useful, especially if the data is formatted into records or more complex structures. Those are more easily processed in native binary representation. Another potential wrinkle is byte order, but I presume that all the devices involved use the same native representation and byte ordering. |
|
Subject: RE: Tech: converting binary data to ASCII From: MartinRyan Date: 29 Oct 08 - 04:15 PM A good time of the year to be dabbling in Hex! Regards |
|
Subject: RE: Tech: converting binary data to ASCII From: Stilly River Sage Date: 29 Oct 08 - 05:19 PM I haven't used Hyperterminal in probably a dozen years. I see it is still in the Accessories list, even after DOS went away, and that one still has a few uses. So what are you doing, throwing tubas at land mines? Why not use accordians instead? Seems a waste to ruin a perfectly good tuba. SRS |
|
Subject: RE: Tech: converting binary data to ASCII From: dick greenhaus Date: 29 Oct 08 - 05:59 PM I'm still not sure exactly what you're trying to convert. Does a string of bits represent a character? If so, how many (8 or 9 is common). Once you've defined what you're looking for, it's a pretty trivial conversion (I'm used to APL, so it would involve a program of maybe a dozen keystrokes.) |
|
Subject: RE: Tech: converting binary data to ASCII From: JohnInKansas Date: 29 Oct 08 - 11:27 PM In common use, the term "binary file" just means that the file contains no - or few - formatting bits, tags, labels, and other "non-essential" information. "All computer files are binary." In the broadest sense, a hard drive contains ONE BINARY FILE. The file is broken up by placing "markers" at fixed intervals within the file, to identify "clusters" whose sole real purpose are to let the computer know "where it is" within the "file." An "index" is used to identify a specific cluster (location on the disk) where an "individual file" begins, and usually each cluster contains additional "markers" that identify which file it contains, which cluster precedes it in that file, and which cluster follows it in that file. Most "files" that are associated with a particular program, or that conform to some "standard format" also contain additional "format information" that gives the program used to read and display the file information on the specific way(s) the bits should be read. While these still are - for the technical purist - binary files, the more specific name that identifies a particular format is generally used, and the "file extensions" provide some bit of "shorthand" for the identification of a particular format. The additional "format bits" allow a program that reads "binary files of format type xxxxx" to interpret the bits in appropriate ways, and to change the way the bits are interpreted within a particular "span of bits" if that is needed for proper display or other use. When a "transducer" or other kind of instrument is used to measure some physical parameter, it's usual for the measuring device to "send" only a "number" corresponding to the value of the parameter being measured. If only one, or a few, "numbers" are needed it should be fairly easy to convert the numbers - each sent as a binary number or as a "character encoded formatted" number like hexadecimal, BCD (binary coded decimal) or an ASCII/ANSI/Unicode "character." If "lots of numbers" are needed, particularly in the case of a "continuous stream of readings," it's common to use a "logging program" to keep the numbers orderly. In one common "format" a specific number of "numbers" (values) is "written" in a single column or in a single row, and the computer receiving the information (i.e. the logging program) "attaches" some bit of identifying information so you'll be able later to know "which numbers came when." In the case of "intermittent readings" each received number/value may have a "timestamp" attached to each reading, when the reading is "logged" into the database file by the "logging program." In some cases just a "sequence number" might be sufficient. If data is "streamed" at some fixed rate, the logging program may timestamp only each "row/column" of numbers, or for high speed data only each "nth row." For a data stream arriving at moderate to high frequencies, the "timestamp" may give GMT or some other "universal time" click for every nth row, and only an "offset from the last universal time" for each row until the next "universal" stamp. The periodic universal time may be called a "synch" marker, while the "offset times" are just "identifiers" of individual "data points" within the span between synchs. If the data values are not individually "marked," the computer (or the user manually) takes 4, 8 or 16 bits and assumes that's a "value" for a data point. If a single bit is "dropped" or one is added by noise in the experimental setup, every value that follows will be mis-read, since the conversion to "useful form" will start and end at the wrong points in the "stream of bits," and each reported value will actually be a mixture of bits from what should have been two different original data values. The "synch marker" can be used to discard data that was misread, and to restart "where the next value starts" so that subsequent information is correctly read. In a University or moderately sized "research facility" it's common for the organization having control of the computer where data will be logged and made accessible for "interpreted use" to provide the "logging program" that individual researchers are expected to use. Such programs appear to commonly allow the user to specify things like byte length, whether data will be in rows or in columns and how many of each, frequency of "tags" and "synch tags" and perhaps other details appropriate for a specific data set. I haven't found much in the way of logging interfaces that are accessible for individual use. Most isolated experimenters appear to write their own. Where a single data kind must be handled, this doesn't appear to be a particularly complex programming task, since the extra features to "generalize for many users with varied data types" can be omitted - but it is a programming task that probably should be done for anything but "trivially simple" data. The benefits of "organizing" your data via an appropriate "logging interface" would justify trying to find someone who's actually "done it" and has the access that would permit "loaning" (or stealing1 for you) a known good program. 1 Within the bounds of ethical conduct, of course. John |
|
Subject: RE: Tech: converting binary data to ASCII From: pavane Date: 30 Oct 08 - 04:30 AM And Dick - NO IT DOES NOT. As I understand, certain number of bits (Maybe 16 or 24) in this stream defines the binary value of a number. Nothing to do with characters. If you have saved it in a file, it will be a DATA file, not a TEXT file. (even if it has the extension .txt) Interpreting the characters as ASCII, as a program like NOTEPAD would do, is meaningless. That's what John is explaining, though in more words. |
|
Subject: RE: Tech: converting binary data to ASCII From: JohnInKansas Date: 30 Oct 08 - 05:57 AM Side Note: From: Stilly River Sage - PM Date: 29 Oct 08 - 05:19 PM I haven't used Hyperterminal in probably a dozen years. I see it is still in the Accessories list, even after DOS went away, and that one still has a few uses. HyperTerminal no longer exists in Vista. Vista Help states: HyperTerminal is no longer part of Windows. There are several options you can use to replace its functionality. If you only need remote shell access, you can use WinRS (Windows Remote Shell), new in Windows Vista. To get help with WinRS, type winrs /? at the command prompt, and then press ENTER. Telnet is a simple, text-based program that you can use to connect to another computer over the Internet. If you previously used HyperTerminal to troubleshoot modem problems, use Phone and Modem Options instead ... The "winrs /?" help leads to the additional: To manage active remote shells or WinRS configuration, use the WinRM tool. The URI alias to manage active shells is shell/cmd. The URI alias for WinRS configuration is winrm/config/winrs. Example usage can be found in the WinRM tool by typing "WinRM -?". There are a number of "new and different" command line functions in Vista; and some old ones, like HyperTerminal, have been dropped. Some new Vista command line functions actually do offer more powerful and precise control of things. Having forgotten anything I may have known about HyperTerminal, I can't readily compare the previous features to what can be done with Vista winrs and WinRM commands. Using "HyperTerminal" as the search string in WinXP Help does show 15 Microsoft Knowledge Base articles that may be of significance for particular uses, with some being specific to particular Windows versions. (One that my curiosity may force me to read (later) implies that HyperTerminal may be subject to errors in transmission of hexadecimal data.) John |
|
Subject: RE: Tech: converting binary data to ASCII From: Paul Burke Date: 30 Oct 08 - 06:25 AM RealTerm is a useful, free, terminal program that can capture a serial input and display it in ASCII, or as a readable number if the word length is fixed or if there's an identifiable header. As for aliasing, if there isn't a hairy antialiasing filter in the accelerometer, you're stuffed as it's already sampled by the time you get it. But it might not be a problem- I don't know much about it, but I'd expect higher- frequency sounds to couple poorly into the ground. In any case, if the higher frequencies contain information, the aliased signal is really as good as the original, as what you're loking for is a characteristic response. There was a technique sometimes used in archaeology before resistometers, magnetometers and GPR became popular, that involved banging on the ground with one pickaxe while listening to the response with your ear on the end of another handle held against the ground. It wasn't dreadfully successful unless there were big cavities below ground. If landmines are involved, please instruct the tuba player NOT to march, otherwise detection by another method could occur. |
|
Subject: RE: Tech: converting binary data to ASCII From: George Seto - af221@chebucto.ns.ca Date: 30 Oct 08 - 07:43 AM Hi TIA. Thanks. So we're working from an actual file. Do you know or have information on that file's format? Those two programs I linked to for you should be able to process the file into ASCII if the data file is straight ASCII. Please note, ASCII is "not" simple. There is the original 7 bit ASCII which is True ASCII for ancients like myself, but then there is 8 bit ASCII where you'd have to know whether the eighth bit is Even Parity/Odd Parity/Null or No Parity. As Dick mentioned, depending on the originating system, it could be binary with low bit first or the other way with high bit first. JohnInKansas is giving heavy tech speak, but it boils down to "find out what the convention is for your binary data". |
|
Subject: RE: Tech: converting binary data to ASCII From: The Fooles Troupe Date: 30 Oct 08 - 07:55 AM Ah George! a fellow "ancient" (old fart!)! - I even remember the DEC10 36 bit 'packed character format' where you had 5x7bit characters packed into a 36 bit "word'.... :0 |
|
Subject: RE: Tech: converting binary data to ASCII From: GUEST,chris Date: 30 Oct 08 - 09:59 AM I'm sure the words are in english???????????????????????????? |
|
Subject: RE: Tech: converting binary data to ASCII From: pavane Date: 30 Oct 08 - 10:24 AM ICL 1900 had four 6-bit characters packed into a 24 bit word. That didn't allow enough different characters, so there were a few "shift" characters which changed the meaning of the following one. George - it is clear that we are NOT dealing with ASCII in any shape or form. We are dealing with BINARY NUMBERS. Example: ASCII is just a way of assigning numeric values to a list of characters, for the purpose of displaying or printing TEXT. Computers cannot (usually) compute in decimal notation, though there are exceptions. Therefore they cannot directly understand numbers that we type at the keyboard. These have to be converted to BINARY before any arithmetic is done. The numeric ASCII code for the number 4 is 52 In binary, this would be represented by 00110100, or in Hexadecimal, 34. This should NOT BE REGARDED as a number, it is just a code to tell the computer to input or output the character "4". The character "a" is hex 61, and "A" is hex 41. Hexadecimal is just a shorthand way of writing binary, using the numbers 0 to 9, and the letters A to F, to represent the 16 possible values which can be held in 4 bits (Called a 'nibble'). A byte contains two nibbles, therefore the hexadecimal code for the value in a byte has two digits or letters. The BINARY number 4 in an 8-bit number is 00000100, or HEX 04 The BINARY number 4 in a 16-bit number is 0000000000000100 (In HEXADECIMAL, this is 00 04) This would be interpreted as the character for 00, followed by the character 04, neither of which is printable. I think 00 might show as a smiley face in a text editor.. The ASCII (Printed) number 69 would be represented as the character 6, followed by the character 9. Therefore (as hex, two bytes 36 39) it would be 0011 0110 0011 1001 As a single binary number, this would be 13881. Nothing like 69 at all! |
|
Subject: RE: Tech: converting binary data to ASCII From: Paul Burke Date: 30 Oct 08 - 10:57 AM this would be 13881. Nothing like 69 at all Have you ever tried 13881? |
|
Subject: RE: Tech: converting binary data to ASCII From: pavane Date: 30 Oct 08 - 11:01 AM Yes, but the noodles were burnt. |
| Share Thread: |