Lexical Analysis With Flex, for Flex 2.5.37

flex

This manual describes flex, a tool for generating programs that perform pattern-matching on text. The manual includes both tutorial and reference sections.

This edition of The flex Manual documents flex version 2.5.37. It was last updated on 22 July 2012.

This manual was written by Vern Paxson, Will Estes and John Millaway.

• Copyright
• Reporting Bugs
• Introduction
• Simple Examples
• Format
• Patterns
• Matching
• Actions
• Generated Scanner
• Start Conditions
• Multiple Input Buffers
• EOF
• Misc Macros
• User Values
• Yacc
• Scanner Options
• Performance
• Cxx
• Reentrant
• Lex and Posix
• Memory Management
• Serialized Tables
• Diagnostics
• Limitations
• Bibliography
• FAQ
• Appendices
• Indices

--- The Detailed Node Listing ---

Format of the Input File

• Definitions Section
• Rules Section
• User Code Section
• Comments in the Input

Scanner Options

• Options for Specifying Filenames
• Options Affecting Scanner Behavior
• Code-Level And API Options
• Options for Scanner Speed and Size
• Debugging Options
• Miscellaneous Options

Reentrant C Scanners

• Reentrant Uses
• Reentrant Overview
• Reentrant Example
• Reentrant Detail
• Reentrant Functions

The Reentrant API in Detail

• Specify Reentrant
• Extra Reentrant Argument
• Global Replacement
• Init and Destroy Functions
• Accessor Methods
• Extra Data
• About yyscan_t

Memory Management

• The Default Memory Management
• Overriding The Default Memory Management
• A Note About yytext And Memory

Serialized Tables

• Creating Serialized Tables
• Loading and Unloading Serialized Tables
• Tables File Format

FAQ

• When was flex born?
• How do I expand backslash-escape sequences in C-style quoted strings?
• Why do flex scanners call fileno if it is not ANSI compatible?
• Does flex support recursive pattern definitions?
• How do I skip huge chunks of input (tens of megabytes) while using flex?
• Flex is not matching my patterns in the same order that I defined them.
• My actions are executing out of order or sometimes not at all.
• How can I have multiple input sources feed into the same scanner at the same time?
• Can I build nested parsers that work with the same input file?
• How can I match text only at the end of a file?
• How can I make REJECT cascade across start condition boundaries?
• Why cant I use fast or full tables with interactive mode?
• How much faster is -F or -f than -C?
• If I have a simple grammar cant I just parse it with flex?
• Why doesn't yyrestart() set the start state back to INITIAL?
• How can I match C-style comments?
• The period isn't working the way I expected.
• Can I get the flex manual in another format?
• Does there exist a "faster" NDFA->DFA algorithm?
• How does flex compile the DFA so quickly?
• How can I use more than 8192 rules?
• How do I abandon a file in the middle of a scan and switch to a new file?
• How do I execute code only during initialization (only before the first scan)?
• How do I execute code at termination?
• Where else can I find help?
• Can I include comments in the "rules" section of the file?
• I get an error about undefined yywrap().
• How can I change the matching pattern at run time?
• How can I expand macros in the input?
• How can I build a two-pass scanner?
• How do I match any string not matched in the preceding rules?
• I am trying to port code from AT&T lex that uses yysptr and yysbuf.
• Is there a way to make flex treat NULL like a regular character?
• Whenever flex can not match the input it says "flex scanner jammed".
• Why doesn't flex have non-greedy operators like perl does?
• Memory leak - 16386 bytes allocated by malloc.
• How do I track the byte offset for lseek()?
• How do I use my own I/O classes in a C++ scanner?
• How do I skip as many chars as possible?
• deleteme00
• Are certain equivalent patterns faster than others?
• Is backing up a big deal?
• Can I fake multi-byte character support?
• deleteme01
• Can you discuss some flex internals?
• unput() messes up yy_at_bol
• The | operator is not doing what I want
• Why can't flex understand this variable trailing context pattern?
• The ^ operator isn't working
• Trailing context is getting confused with trailing optional patterns
• Is flex GNU or not?
• ERASEME53
• I need to scan if-then-else blocks and while loops
• ERASEME55
• ERASEME56
• ERASEME57
• Is there a repository for flex scanners?
• How can I conditionally compile or preprocess my flex input file?
• Where can I find grammars for lex and yacc?
• I get an end-of-buffer message for each character scanned.
• unnamed-faq-62
• unnamed-faq-63
• unnamed-faq-64
• unnamed-faq-65
• unnamed-faq-66
• unnamed-faq-67
• unnamed-faq-68
• unnamed-faq-69
• unnamed-faq-70
• unnamed-faq-71
• unnamed-faq-72
• unnamed-faq-73
• unnamed-faq-74
• unnamed-faq-75
• unnamed-faq-76
• unnamed-faq-77
• unnamed-faq-78
• unnamed-faq-79
• unnamed-faq-80
• unnamed-faq-81
• unnamed-faq-82
• unnamed-faq-83
• unnamed-faq-84
• unnamed-faq-85
• unnamed-faq-86
• unnamed-faq-87
• unnamed-faq-88
• unnamed-faq-90
• unnamed-faq-91
• unnamed-faq-92
• unnamed-faq-93
• unnamed-faq-94
• unnamed-faq-95
• unnamed-faq-96
• unnamed-faq-97
• unnamed-faq-98
• unnamed-faq-99
• unnamed-faq-100
• unnamed-faq-101
• What is the difference between YYLEX_PARAM and YY_DECL?
• Why do I get "conflicting types for yylex" error?
• How do I access the values set in a Flex action from within a Bison action?

Appendices

• Makefiles and Flex
• Bison Bridge
• M4 Dependency
• Common Patterns

Indices

• Concept Index
• Index of Functions and Macros
• Index of Variables
• Index of Data Types
• Index of Hooks
• Index of Scanner Options