Fields inherited from class org.apache.lucene.analysis.Tokenizer
input
Constructor Summary
CustomWhiteSpaceTokenizer(java.io.Reader in,
java.util.Set whiteSpaceChars)
Construct a new WhitespaceTokenizer.
Method Summary
protected boolean
isTokenChar(char c)
Collects only characters which do not satisfy
Character.isWhitespace(char), and are not in the
whiteSpaceCharsToRemove set.
static java.util.Set
makeCharWhiteSpaceSet(char[] charsToTreatAsWhiteSpace)
Builds a Set from an array of chars to treat as whitespace, appropriate
for passing into the CustomWhiteSpaceTokenizer constructor.
Methods inherited from class org.apache.lucene.analysis.CharTokenizer
next, normalize, reset
Methods inherited from class org.apache.lucene.analysis.Tokenizer
close
Methods inherited from class org.apache.lucene.analysis.TokenStream
Copyright: (c) 2004-2006 Mayo Foundation for Medical Education and Research (MFMER). All rights reserved. MAYO, MAYO CLINIC, and the triple-shield Mayo logo are trademarks and service marks of MFMER.