Re: A86: Tokenization
[Prev][Next][Index][Thread]
Re: A86: Tokenization
At 08:17 PM 6/24/98 -0400, you wrote:
>
>Cool... A good way to do it is to make a sqrtPARSE prog and then just set a
>breakpoint and look at the ! or # or whatever object. This is what I've
found
>so far (may be redundant):
>
I was just writing a basic program with a command, then running it and
looking at the tokenized version.
>Decimal numbers tokenize to a 'D' followed by a null-terminated string of the
>number exactly as it was typed. '+'->$60 '-'->$61, '*'->$70, '/'->$71...
And strings tokenize to $27, followed by the null-terminated string.
>
>I know it's not that much but i wasn't too concerned w/ the actual
>tokenization
>
Me neither...yet. Actually, I'm more interested in looking at how the
normal parser handles stuff, so I needed to know some of the codes so I
could figure out what each part was supposed to do. It seems kind of
pointless to tokenize some of this stuff, like all the one character
operators. It seems kind of silly to change '(' to $10, '+' to $60, etc. I
mean, it doesn't compress the program at all, it just makes the code that
handles it harder to read. Oh well...
--Joshua
References: