README (2632:1bb2f91485ea) README (4479:61d3ed46e373)
1PLY (Python Lex-Yacc) Version 1.2 (November 27, 2002)
1PLY (Python Lex-Yacc) Version 2.3 (February 18, 2007)
2
2
3David M. Beazley
4Department of Computer Science
5University of Chicago
6Chicago, IL 60637
7beazley@cs.uchicago.edu
3David M. Beazley (dave@dabeaz.com)
8
4
9Copyright (C) 2001 David M. Beazley
5Copyright (C) 2001-2007 David M. Beazley
10
6
11$Header: /home/stever/bk/newmem2/ext/ply/README 1.1 03/06/06 14:53:34-00:00 stever@ $
12
13This library is free software; you can redistribute it and/or
14modify it under the terms of the GNU Lesser General Public
15License as published by the Free Software Foundation; either
16version 2.1 of the License, or (at your option) any later version.
17
18This library is distributed in the hope that it will be useful,
19but WITHOUT ANY WARRANTY; without even the implied warranty of
20MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU

--- 26 unchanged lines hidden (view full) ---

47 precedence specifiers, and moderately ambiguous grammars.
48
49 - Parsing is based on LR-parsing which is fast, memory efficient,
50 better suited to large grammars, and which has a number of nice
51 properties when dealing with syntax errors and other parsing problems.
52 Currently, PLY builds its parsing tables using the SLR algorithm which
53 is slightly weaker than LALR(1) used in traditional yacc.
54
7This library is free software; you can redistribute it and/or
8modify it under the terms of the GNU Lesser General Public
9License as published by the Free Software Foundation; either
10version 2.1 of the License, or (at your option) any later version.
11
12This library is distributed in the hope that it will be useful,
13but WITHOUT ANY WARRANTY; without even the implied warranty of
14MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU

--- 26 unchanged lines hidden (view full) ---

41 precedence specifiers, and moderately ambiguous grammars.
42
43 - Parsing is based on LR-parsing which is fast, memory efficient,
44 better suited to large grammars, and which has a number of nice
45 properties when dealing with syntax errors and other parsing problems.
46 Currently, PLY builds its parsing tables using the SLR algorithm which
47 is slightly weaker than LALR(1) used in traditional yacc.
48
55 - Like John Aycock's excellent SPARK toolkit, PLY uses Python
56 reflection to build lexers and parsers. This greatly simplifies
57 the task of parser construction since it reduces the number of files
58 and eliminates the need to run a separate lex/yacc tool before
59 running your program.
49 - PLY uses Python introspection features to build lexers and parsers.
50 This greatly simplifies the task of parser construction since it reduces
51 the number of files and eliminates the need to run a separate lex/yacc
52 tool before running your program.
60
61 - PLY can be used to build parsers for "real" programming languages.
62 Although it is not ultra-fast due to its Python implementation,
63 PLY can be used to parse grammars consisting of several hundred
64 rules (as might be found for a language like C). The lexer and LR
65 parser are also reasonably efficient when parsing typically
66 sized programs.
67

--- 4 unchanged lines hidden (view full) ---

72assembly code for the SPARC processor. Because of this, the current
73implementation has been extensively tested and debugged. In addition,
74most of the API and error checking steps have been adapted to address
75common usability problems.
76
77How to Use
78==========
79
53
54 - PLY can be used to build parsers for "real" programming languages.
55 Although it is not ultra-fast due to its Python implementation,
56 PLY can be used to parse grammars consisting of several hundred
57 rules (as might be found for a language like C). The lexer and LR
58 parser are also reasonably efficient when parsing typically
59 sized programs.
60

--- 4 unchanged lines hidden (view full) ---

65assembly code for the SPARC processor. Because of this, the current
66implementation has been extensively tested and debugged. In addition,
67most of the API and error checking steps have been adapted to address
68common usability problems.
69
70How to Use
71==========
72
80PLY consists of two files : lex.py and yacc.py. To use the system,
81simply copy these files to your project and import them like standard
82Python modules.
73PLY consists of two files : lex.py and yacc.py. These are contained
74within the 'ply' directory which may also be used as a Python package.
75To use PLY, simply copy the 'ply' directory to your project and import
76lex and yacc from the associated 'ply' package. For example:
83
77
78 import ply.lex as lex
79 import ply.yacc as yacc
80
81Alternatively, you can copy just the files lex.py and yacc.py
82individually and use them as modules. For example:
83
84 import lex
85 import yacc
86
87The file setup.py can be used to install ply using distutils.
88
84The file doc/ply.html contains complete documentation on how to use
85the system.
86
87The example directory contains several different examples including a
89The file doc/ply.html contains complete documentation on how to use
90the system.
91
92The example directory contains several different examples including a
88PLY specification for ANSI C as given in K&R 2nd Ed. Note: To use
89the examples, you will need to copy the lex.py and yacc.py files to
90the example directory.
93PLY specification for ANSI C as given in K&R 2nd Ed.
91
92A simple example is found at the end of this document
93
94Requirements
95============
94
95A simple example is found at the end of this document
96
97Requirements
98============
96PLY requires the use of Python 2.0 or greater. It should work on
97just about any platform.
99PLY requires the use of Python 2.1 or greater. However, you should
100use the latest Python release if possible. It should work on just
101about any platform. PLY has been tested with both CPython and Jython.
102However, it does not seem to work with IronPython.
98
99Resources
100=========
103
104Resources
105=========
101
102More information about PLY can be obtained on the PLY webpage at:
103
106More information about PLY can be obtained on the PLY webpage at:
107
104 http://systems.cs.uchicago.edu/ply
108 http://www.dabeaz.com/ply
105
106For a detailed overview of parsing theory, consult the excellent
107book "Compilers : Principles, Techniques, and Tools" by Aho, Sethi, and
108Ullman. The topics found in "Lex & Yacc" by Levine, Mason, and Brown
109may also be useful.
110
109
110For a detailed overview of parsing theory, consult the excellent
111book "Compilers : Principles, Techniques, and Tools" by Aho, Sethi, and
112Ullman. The topics found in "Lex & Yacc" by Levine, Mason, and Brown
113may also be useful.
114
111Given that this is the first release, I welcome your comments on how
112to improve the current implementation. See the TODO file for things that
113still need to be done.
115A Google group for PLY can be found at
114
116
117 http://groups.google.com/group/ply-hack
118
115Acknowledgments
116===============
119Acknowledgments
120===============
117
118A special thanks is in order for all of the students in CS326 who
119suffered through about 25 different versions of these tools :-).
120
121A special thanks is in order for all of the students in CS326 who
122suffered through about 25 different versions of these tools :-).
123
124The CHANGES file acknowledges those who have contributed patches.
125
126Elias Ioup did the first implementation of LALR(1) parsing in PLY-1.x.
127Andrew Waters and Markus Schoepflin were instrumental in reporting bugs
128and testing a revised LALR(1) implementation for PLY-2.0.
129
130Special Note for PLY-2.x
131========================
132PLY-2.0 is the first in a series of PLY releases that will be adding a
133variety of significant new features. The first release in this series
134(Ply-2.0) should be 100% compatible with all previous Ply-1.x releases
135except for the fact that Ply-2.0 features a correct implementation of
136LALR(1) table generation.
137
138If you have suggestions for improving PLY in future 2.x releases, please
139contact me. - Dave
140
121Example
122=======
123
141Example
142=======
143
124Here is a simple example showing a PLY implementation of a calculator with variables.
144Here is a simple example showing a PLY implementation of a calculator
145with variables.
125
126# -----------------------------------------------------------------------------
127# calc.py
128#
129# A simple calculator with variables.
130# -----------------------------------------------------------------------------
131
132tokens = (

--- 22 unchanged lines hidden (view full) ---

155 t.value = 0
156 return t
157
158# Ignored characters
159t_ignore = " \t"
160
161def t_newline(t):
162 r'\n+'
146
147# -----------------------------------------------------------------------------
148# calc.py
149#
150# A simple calculator with variables.
151# -----------------------------------------------------------------------------
152
153tokens = (

--- 22 unchanged lines hidden (view full) ---

176 t.value = 0
177 return t
178
179# Ignored characters
180t_ignore = " \t"
181
182def t_newline(t):
183 r'\n+'
163 t.lineno += t.value.count("\n")
184 t.lexer.lineno += t.value.count("\n")
164
165def t_error(t):
166 print "Illegal character '%s'" % t.value[0]
185
186def t_error(t):
187 print "Illegal character '%s'" % t.value[0]
167 t.skip(1)
188 t.lexer.skip(1)
168
169# Build the lexer
189
190# Build the lexer
170import lex
191import ply.lex as lex
171lex.lex()
172
173# Precedence rules for the arithmetic operators
174precedence = (
175 ('left','PLUS','MINUS'),
176 ('left','TIMES','DIVIDE'),
177 ('right','UMINUS'),
178 )
179
180# dictionary of names (for storing variables)
181names = { }
182
192lex.lex()
193
194# Precedence rules for the arithmetic operators
195precedence = (
196 ('left','PLUS','MINUS'),
197 ('left','TIMES','DIVIDE'),
198 ('right','UMINUS'),
199 )
200
201# dictionary of names (for storing variables)
202names = { }
203
183def p_statement_assign(t):
204def p_statement_assign(p):
184 'statement : NAME EQUALS expression'
205 'statement : NAME EQUALS expression'
185 names[t[1]] = t[3]
206 names[p[1]] = p[3]
186
207
187def p_statement_expr(t):
208def p_statement_expr(p):
188 'statement : expression'
209 'statement : expression'
189 print t[1]
210 print p[1]
190
211
191def p_expression_binop(t):
212def p_expression_binop(p):
192 '''expression : expression PLUS expression
193 | expression MINUS expression
194 | expression TIMES expression
195 | expression DIVIDE expression'''
213 '''expression : expression PLUS expression
214 | expression MINUS expression
215 | expression TIMES expression
216 | expression DIVIDE expression'''
196 if t[2] == '+' : t[0] = t[1] + t[3]
197 elif t[2] == '-': t[0] = t[1] - t[3]
198 elif t[2] == '*': t[0] = t[1] * t[3]
199 elif t[2] == '/': t[0] = t[1] / t[3]
217 if p[2] == '+' : p[0] = p[1] + p[3]
218 elif p[2] == '-': p[0] = p[1] - p[3]
219 elif p[2] == '*': p[0] = p[1] * p[3]
220 elif p[2] == '/': p[0] = p[1] / p[3]
200
221
201def p_expression_uminus(t):
222def p_expression_uminus(p):
202 'expression : MINUS expression %prec UMINUS'
223 'expression : MINUS expression %prec UMINUS'
203 t[0] = -t[2]
224 p[0] = -p[2]
204
225
205def p_expression_group(t):
226def p_expression_group(p):
206 'expression : LPAREN expression RPAREN'
227 'expression : LPAREN expression RPAREN'
207 t[0] = t[2]
228 p[0] = p[2]
208
229
209def p_expression_number(t):
230def p_expression_number(p):
210 'expression : NUMBER'
231 'expression : NUMBER'
211 t[0] = t[1]
232 p[0] = p[1]
212
233
213def p_expression_name(t):
234def p_expression_name(p):
214 'expression : NAME'
215 try:
235 'expression : NAME'
236 try:
216 t[0] = names[t[1]]
237 p[0] = names[p[1]]
217 except LookupError:
238 except LookupError:
218 print "Undefined name '%s'" % t[1]
219 t[0] = 0
239 print "Undefined name '%s'" % p[1]
240 p[0] = 0
220
241
221def p_error(t):
222 print "Syntax error at '%s'" % t.value
242def p_error(p):
243 print "Syntax error at '%s'" % p.value
223
244
224import yacc
245import ply.yacc as yacc
225yacc.yacc()
226
227while 1:
228 try:
229 s = raw_input('calc > ')
230 except EOFError:
231 break
232 yacc.parse(s)
233
234
246yacc.yacc()
247
248while 1:
249 try:
250 s = raw_input('calc > ')
251 except EOFError:
252 break
253 yacc.parse(s)
254
255
256Bug Reports and Patches
257=======================
258Because of the extremely specialized and advanced nature of PLY, I
259rarely spend much time working on it unless I receive very specific
260bug-reports and/or patches to fix problems. I also try to incorporate
261submitted feature requests and enhancements into each new version. To
262contact me about bugs and/or new features, please send email to
263dave@dabeaz.com.
235
264
265In addition there is a Google group for discussing PLY related issues at
236
266
267 http://groups.google.com/group/ply-hack
268
269-- Dave
237
238
239
240
270
271
272
273
241
242
243
244
245
246
274
275
276
277
278
247
248
249