CHANGES revision 2632
12632Sstever@eecs.umich.eduVersion 1.3 22632Sstever@eecs.umich.edu------------------------------ 32632Sstever@eecs.umich.edu12/10/02: jmdyck 42632Sstever@eecs.umich.edu Various minor adjustments to the code that Dave checked in today. 52632Sstever@eecs.umich.edu Updated test/yacc_{inf,unused}.exp to reflect today's changes. 62632Sstever@eecs.umich.edu 72632Sstever@eecs.umich.edu12/10/02: beazley 82632Sstever@eecs.umich.edu Incorporated a variety of minor bug fixes to empty production 92632Sstever@eecs.umich.edu handling and infinite recursion checking. Contributed by 102632Sstever@eecs.umich.edu Michael Dyck. 112632Sstever@eecs.umich.edu 122632Sstever@eecs.umich.edu12/10/02: beazley 132632Sstever@eecs.umich.edu Removed bogus recover() method call in yacc.restart() 142632Sstever@eecs.umich.edu 152632Sstever@eecs.umich.eduVersion 1.2 162632Sstever@eecs.umich.edu------------------------------ 172632Sstever@eecs.umich.edu11/27/02: beazley 182632Sstever@eecs.umich.edu Lexer and parser objects are now available as an attribute 192632Sstever@eecs.umich.edu of tokens and slices respectively. For example: 202632Sstever@eecs.umich.edu 212632Sstever@eecs.umich.edu def t_NUMBER(t): 222632Sstever@eecs.umich.edu r'\d+' 232632Sstever@eecs.umich.edu print t.lexer 242632Sstever@eecs.umich.edu 252632Sstever@eecs.umich.edu def p_expr_plus(t): 262632Sstever@eecs.umich.edu 'expr: expr PLUS expr' 272632Sstever@eecs.umich.edu print t.lexer 282632Sstever@eecs.umich.edu print t.parser 292632Sstever@eecs.umich.edu 302632Sstever@eecs.umich.edu This can be used for state management (if needed). 312632Sstever@eecs.umich.edu 322632Sstever@eecs.umich.edu10/31/02: beazley 332632Sstever@eecs.umich.edu Modified yacc.py to work with Python optimize mode. To make 342632Sstever@eecs.umich.edu this work, you need to use 352632Sstever@eecs.umich.edu 362632Sstever@eecs.umich.edu yacc.yacc(optimize=1) 372632Sstever@eecs.umich.edu 382632Sstever@eecs.umich.edu Furthermore, you need to first run Python in normal mode 392632Sstever@eecs.umich.edu to generate the necessary parsetab.py files. After that, 402632Sstever@eecs.umich.edu you can use python -O or python -OO. 412632Sstever@eecs.umich.edu 422632Sstever@eecs.umich.edu Note: optimized mode turns off a lot of error checking. 432632Sstever@eecs.umich.edu Only use when you are sure that your grammar is working. 442632Sstever@eecs.umich.edu Make sure parsetab.py is up to date! 452632Sstever@eecs.umich.edu 462632Sstever@eecs.umich.edu10/30/02: beazley 472632Sstever@eecs.umich.edu Added cloning of Lexer objects. For example: 482632Sstever@eecs.umich.edu 492632Sstever@eecs.umich.edu import copy 502632Sstever@eecs.umich.edu l = lex.lex() 512632Sstever@eecs.umich.edu lc = copy.copy(l) 522632Sstever@eecs.umich.edu 532632Sstever@eecs.umich.edu l.input("Some text") 542632Sstever@eecs.umich.edu lc.input("Some other text") 552632Sstever@eecs.umich.edu ... 562632Sstever@eecs.umich.edu 572632Sstever@eecs.umich.edu This might be useful if the same "lexer" is meant to 582632Sstever@eecs.umich.edu be used in different contexts---or if multiple lexers 592632Sstever@eecs.umich.edu are running concurrently. 602632Sstever@eecs.umich.edu 612632Sstever@eecs.umich.edu10/30/02: beazley 622632Sstever@eecs.umich.edu Fixed subtle bug with first set computation and empty productions. 632632Sstever@eecs.umich.edu Patch submitted by Michael Dyck. 642632Sstever@eecs.umich.edu 652632Sstever@eecs.umich.edu10/30/02: beazley 662632Sstever@eecs.umich.edu Fixed error messages to use "filename:line: message" instead 672632Sstever@eecs.umich.edu of "filename:line. message". This makes error reporting more 682632Sstever@eecs.umich.edu friendly to emacs. Patch submitted by Fran�ois Pinard. 692632Sstever@eecs.umich.edu 702632Sstever@eecs.umich.edu10/30/02: beazley 712632Sstever@eecs.umich.edu Improvements to parser.out file. Terminals and nonterminals 722632Sstever@eecs.umich.edu are sorted instead of being printed in random order. 732632Sstever@eecs.umich.edu Patch submitted by Fran�ois Pinard. 742632Sstever@eecs.umich.edu 752632Sstever@eecs.umich.edu10/30/02: beazley 762632Sstever@eecs.umich.edu Improvements to parser.out file output. Rules are now printed 772632Sstever@eecs.umich.edu in a way that's easier to understand. Contributed by Russ Cox. 782632Sstever@eecs.umich.edu 792632Sstever@eecs.umich.edu10/30/02: beazley 802632Sstever@eecs.umich.edu Added 'nonassoc' associativity support. This can be used 812632Sstever@eecs.umich.edu to disable the chaining of operators like a < b < c. 822632Sstever@eecs.umich.edu To use, simply specify 'nonassoc' in the precedence table 832632Sstever@eecs.umich.edu 842632Sstever@eecs.umich.edu precedence = ( 852632Sstever@eecs.umich.edu ('nonassoc', 'LESSTHAN', 'GREATERTHAN'), # Nonassociative operators 862632Sstever@eecs.umich.edu ('left', 'PLUS', 'MINUS'), 872632Sstever@eecs.umich.edu ('left', 'TIMES', 'DIVIDE'), 882632Sstever@eecs.umich.edu ('right', 'UMINUS'), # Unary minus operator 892632Sstever@eecs.umich.edu ) 902632Sstever@eecs.umich.edu 912632Sstever@eecs.umich.edu Patch contributed by Russ Cox. 922632Sstever@eecs.umich.edu 932632Sstever@eecs.umich.edu10/30/02: beazley 942632Sstever@eecs.umich.edu Modified the lexer to provide optional support for Python -O and -OO 952632Sstever@eecs.umich.edu modes. To make this work, Python *first* needs to be run in 962632Sstever@eecs.umich.edu unoptimized mode. This reads the lexing information and creates a 972632Sstever@eecs.umich.edu file "lextab.py". Then, run lex like this: 982632Sstever@eecs.umich.edu 992632Sstever@eecs.umich.edu # module foo.py 1002632Sstever@eecs.umich.edu ... 1012632Sstever@eecs.umich.edu ... 1022632Sstever@eecs.umich.edu lex.lex(optimize=1) 1032632Sstever@eecs.umich.edu 1042632Sstever@eecs.umich.edu Once the lextab file has been created, subsequent calls to 1052632Sstever@eecs.umich.edu lex.lex() will read data from the lextab file instead of using 1062632Sstever@eecs.umich.edu introspection. In optimized mode (-O, -OO) everything should 1072632Sstever@eecs.umich.edu work normally despite the loss of doc strings. 1082632Sstever@eecs.umich.edu 1092632Sstever@eecs.umich.edu To change the name of the file 'lextab.py' use the following: 1102632Sstever@eecs.umich.edu 1112632Sstever@eecs.umich.edu lex.lex(lextab="footab") 1122632Sstever@eecs.umich.edu 1132632Sstever@eecs.umich.edu (this creates a file footab.py) 1142632Sstever@eecs.umich.edu 1152632Sstever@eecs.umich.edu 1162632Sstever@eecs.umich.eduVersion 1.1 October 25, 2001 1172632Sstever@eecs.umich.edu------------------------------ 1182632Sstever@eecs.umich.edu 1192632Sstever@eecs.umich.edu10/25/01: beazley 1202632Sstever@eecs.umich.edu Modified the table generator to produce much more compact data. 1212632Sstever@eecs.umich.edu This should greatly reduce the size of the parsetab.py[c] file. 1222632Sstever@eecs.umich.edu Caveat: the tables still need to be constructed so a little more 1232632Sstever@eecs.umich.edu work is done in parsetab on import. 1242632Sstever@eecs.umich.edu 1252632Sstever@eecs.umich.edu10/25/01: beazley 1262632Sstever@eecs.umich.edu There may be a possible bug in the cycle detector that reports errors 1272632Sstever@eecs.umich.edu about infinite recursion. I'm having a little trouble tracking it 1282632Sstever@eecs.umich.edu down, but if you get this problem, you can disable the cycle 1292632Sstever@eecs.umich.edu detector as follows: 1302632Sstever@eecs.umich.edu 1312632Sstever@eecs.umich.edu yacc.yacc(check_recursion = 0) 1322632Sstever@eecs.umich.edu 1332632Sstever@eecs.umich.edu10/25/01: beazley 1342632Sstever@eecs.umich.edu Fixed a bug in lex.py that sometimes caused illegal characters to be 1352632Sstever@eecs.umich.edu reported incorrectly. Reported by Sverre J�rgensen. 1362632Sstever@eecs.umich.edu 1372632Sstever@eecs.umich.edu7/8/01 : beazley 1382632Sstever@eecs.umich.edu Added a reference to the underlying lexer object when tokens are handled by 1392632Sstever@eecs.umich.edu functions. The lexer is available as the 'lexer' attribute. This 1402632Sstever@eecs.umich.edu was added to provide better lexing support for languages such as Fortran 1412632Sstever@eecs.umich.edu where certain types of tokens can't be conveniently expressed as regular 1422632Sstever@eecs.umich.edu expressions (and where the tokenizing function may want to perform a 1432632Sstever@eecs.umich.edu little backtracking). Suggested by Pearu Peterson. 1442632Sstever@eecs.umich.edu 1452632Sstever@eecs.umich.edu6/20/01 : beazley 1462632Sstever@eecs.umich.edu Modified yacc() function so that an optional starting symbol can be specified. 1472632Sstever@eecs.umich.edu For example: 1482632Sstever@eecs.umich.edu 1492632Sstever@eecs.umich.edu yacc.yacc(start="statement") 1502632Sstever@eecs.umich.edu 1512632Sstever@eecs.umich.edu Normally yacc always treats the first production rule as the starting symbol. 1522632Sstever@eecs.umich.edu However, if you are debugging your grammar it may be useful to specify 1532632Sstever@eecs.umich.edu an alternative starting symbol. Idea suggested by Rich Salz. 1542632Sstever@eecs.umich.edu 1552632Sstever@eecs.umich.eduVersion 1.0 June 18, 2001 1562632Sstever@eecs.umich.edu-------------------------- 1572632Sstever@eecs.umich.eduInitial public offering 1582632Sstever@eecs.umich.edu 159