Zystkmt

Having an issue with my parser and I just can't wrap my head around it. I keep getting the error: parser.y:79.33-41: symbol character is used, but is not defined as a token and has no rules. I've put the line in ** **.

Below is my parser code:

%token          PLUS TIMES DIVIDE SUBTRACT BRA KET EQUALS NOT_EQUAL LESS_THAN GREATER_THAN GREATER_THAN_OR_EQUAL

%token          COMA SEMICOLON ARROW DECIMAL ENDDO ENDFOR ENDIF ENDP ENDWHILE ELSE CODE OF TYPE DECLARATIONS CHARACTER 

%token          INTEGER REAL IF THEN DO WHILE FOR IS BY TO WRITE NEWLINE READ NOT AND OR ID LESS_THAN_OR_EQUAL APOSTROPHE 



%%

Program : block 

            | ENDP 

            | ID

    ;

block : DECLARATIONS declaration_block  CODE  statement_list 

        | CODE statement_list

    ;

declaration_block : ID OF TYPE SEMICOLON 

                    | ID_list OF TYPE SEMICOLON                     

                    | ID_list OF TYPE SEMICOLON declaration_block

    ;               

ID_list :           ID 

                    | ID COMA ID_list

    ;               

type :  CHARACTER 

        | INTEGER 

        | REAL

    ;   

statement_list : statement 

                    | statement_list SEMICOLON statement

    ;               

statement : assignment_statement 

                | if_statement

                | do_statement 

                | while_statement 

                | for_statement 

                | write_statement 

                | read_statement

    ;

assignment_statement : expression ARROW ID

    ;

if_statement : IF  conditional  THEN  statement_list  ELSE  statement_list  ENDIF

    ;

do_statement : DO statement_list WHILE  conditional  ENDDO

    ;

while_statement : WHILE conditional  DO statement_list  ENDWHILE

    ;

for_statement : FOR  ID  IS  expression  BY  expression  TO  expression  DO  statement_list ENDFOR

    ;

write_statement : WRITE BRA output_list KET  write_statement NEWLINE

    ;

read_statement : READ BRA ID KET

    ;

output_list : value 

                | output_list COMA value

    ;

conditional : expression comparator expression 

                | NOT conditional               

                | expression comparator expression AND conditional          

                | expression comparator expression OR conditional           

    ;

comparator : EQUALS 

                | NOT_EQUAL 

                | LESS_THAN 

                | GREATER_THAN 

                | LESS_THAN_OR_EQUAL 

                | GREATER_THAN_OR_EQUAL

    ;               

expression : term 

                | expression PLUS term 

                | expression SUBTRACT term

    ;           

term : value 

        | term TIMES value 

        | term DIVIDE value

    ;   

value : ID 

            | constant 

            | BRA expression KET

    ;       

constant : number_constant 

            | character_constant

    ;       

**character_constant : APOSTROPHE character APOSTROPHE**

    ;

number_constant : INTEGER 

                    | SUBTRACT INTEGER 

                    | SUBTRACT INTEGER DECIMAL INTEGER 

                    | INTEGER DECIMAL INTEGER

    ;           

%%

Below is my lexical analyser:

%{

#ifdef PRINT

#define TOKEN(i) printf("Token: " #i "n");

#else

#define TOKEN(i) return (i);

#endif

%}



delim           [ rnt]

ws              {delim}+

digit           [0-9]

character       [a-zA-Z]

INTEGER         {digit}+

id              {character}({character}|{digit})*

character_constant ('{character}')





%%

{ws}            ; /* Do Nothing */



"+"             TOKEN(PLUS)

"*"             TOKEN(TIMES)

"/"             TOKEN(DIVIDE)

"-"             TOKEN(SUBTRACT)

"("             TOKEN(BRA)

")"             TOKEN(KET)

...

"'"             TOKEN(APOSTROPHE)

ENDP            TOKEN(ENDP)

CODE            TOKEN(CODE)

OF              TOKEN(OF)

TYPE            TOKEN(TYPE)

DECLARATIONS    TOKEN(DECLARATIONS)

character       TOKEN(CHARACTER)

{INTEGER}       TOKEN(INTEGER)

...

{id}            TOKEN(ID)

{character_constant} TOKEN(character_constant)

%%

The spelling is correct throughout my files from my BNF to my parser, does anyone know what seems to be the issue here. I've tried changing it to letter instead of character but that just gives me another error.

The fault is that the single character cannot be differentiated from an identifier in the lexer. You have correctly added a lexer rule to match it which returns a token named character_constant, however you have also put a parser rule for character constant which is not needed, as its already matched in the lexer. The lexer should not be returning an APOSTROPHE as a token as it would have been matched in the character_constant token.

You should just delete the rule from bison for character_constant and add character_constant to the list of tokens matched by the lexer in the %token declaration.

I'd normally recommend using uppercase for the token names to avoid any ambiguity of which is a terminal and which a non-terminal.

The message means exactly what it says. You use the symbol character but you have neither declared it to be a token nor have you provided a production for it. So it is neither a terminal nor a non-terminal, and the parser generator complains when it is used.

What happens in your lexer is not visible to the parser generator. So the fact that you have (pointlessly, IMHO) created a lexer definition of {character} is only of interest inside the lexer file. (It's pointless because you could equivalently use the Posix character class [[:alpha:]] which has a clear meaning and is just as readable as {character}.) And really the parser doesn't need to know anything about how the lexer comes up with the token code value. So when you declare a token in the parser definition, the parser generator puts a definition for that symbol in a header file; the lexer #includes that header file, and that lets it use that symbol as a return value for the parser. No other communication is necessary.